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Introduction 


This  report  describes  a  variety  of  objective  and  subjective  measures  of  performance 
specifically  designed  for  hospital  emergency  departments  (EDs).  Dynamics  Research  Corporation 
(DRC),  in  cooperation  with  Rhode  Island  Hospital  (RIH),  is  developing  the  data  collection 
instruments  to  evaluate  the  Emergency  Team  Coordination  Course  (  ETCC™),  a  training  course 
in  the  principles  and  specific  behaviors  of  effective  teamwork  designed  for  emergency  department 
personnel.  Because  ETCC™  is  based  on  a  philosophy  of  evaluation-based  instruction,  the  content 
and  scope  of  the  course  evaluation  instruments  are  a  central  concern  for  validation  of  the  course. 
A  complete  description  of  the  antecedents  of  the  course  and  its  goals  and  content  are  described  in 
Simon,  Morey,  Locke  and  Blair  (1995)1.  A  brief  summary  is  provided  here. 

A  review  of  the  medical  malpractice  literature  suggested  that  teamwork  errors  are 
implicated  in  emergency  medicine  misadventures.  Field  observations  in  emergency  departments 
provided  examples  of  team  processes  such  as  communication,  workload  management,  team 
building  and  leadership,  planning,  and  decision-making  wherein  team  effectiveness  and  efficiency 
could  be  improved.  These  field  observations,  supplemented  with  information  on  emergency 
department  operations,  resulted  in  a  structure  of  essential  team  functions  referred  to  as  the 
emergency  Team  Dimensions  (TDs).  Specifications  of  superior,  average,  and  very  poor  team 
behavior  were  developed  for  each  of  the  five  TDs  as  behaviorally  anchored  rating  scales  (BARS). 
These  provided  a  basis  for  a  suite  of  subjective  and  objective  team  assessment  instruments 
developed  earlier  in  the  project.  The  one-day  ETCC™  training  program  was  developed  that 
included  lecture,  discussion,  and  practical  exercises  to  teach  team  skills. 

This  report  summarizes  the  results  of  pilot  tests  of  the  original  suite  of  instruments,  and 
new  instruments  developed  to  fulfill  specific  measurement  needs.  The  pilot  tests  were  undertaken 
at  the  Davol  adult  emergency  department  of  RIH.  Note  that  the  ETCC™  curriculum  was  not 
taught  to  Davol  staff.  However,  the  measures  and  instruments  described  in  this  report  assess  ED 
operations  and  staff  processes  that  exist  independent  of  the  training.  The  training  is  designed  to 
improve  the  quality  of  these  operations  and  processes.  The  training  is  not  required  as  a 
prerequisite  to  assessing  the  adequacy  and  performance  of  the  instruments  or  measures.  This 
design  consideration  permits  the  use  of  the  instruments  to  develop  baseline  statistics  prior  to  the 
training. 

After  introducing  and  describing  the  suite  of  measurement  instruments  in  general  terms, 
this  report  presents  a  detailed  description  of  the  instruments  developed  and  data  related  to  their 


1  Simon,  R.,  Morey,  J.  C.,  Locke,  A.  M.,  &  Blair,  E.  Emergency  team  coordination  course™ 
phase  one  report.  (Final  Report).  Wilmington,  MA:  Dynamics  Research  Corporation,  1995 
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administration  and  scoring.  The  next  portion  of  the  report  provides  descriptions  of  the  manpower 
requirements  to  collect  the  data  at  a  typical  ETCC™  validation  site.  Course  validation  will 
consist  of  one  or  more  pretraining  measurement  episodes  lasting  30  days  followed  by  one  or  more 
posttraining  measurement  episodes.  Collection  of  selected  operational  data  such  as  length  of  stay 
and  quality  assurance  indicators  is  planned  for  the  ten  months  surrounding  the  three  months  of  full 
measurement  suite  data  collection. 


Evaluation  Concept 

The  measurement  instruments  developed  to  assess  the  effectiveness  of  the  team  training 
program  were  designed  to  collect  both  subjective  data  (attitudes  and  opinion  ratings)  from 
hospital  staff  and  patients  and  objective  performance  data  (i.e.,  data  reflecting  the  operational 
performance  of  the  emergency  department).  The  measurement  philosophy  is  to  conduct  a  broad 
assessment  of  the  impact  of  the  teamwork  training  by  examining  changes  in  staff  and  patients' 
attitudes,  the  behavior  of  teams,  and  the  operational  performance  of  the  ED.  It  is  unlikely  that  one 
type  of  measure  will  provide  sufficient  evidence  of  the  course's  impact,  but  collectively  the 
measures  will  provide  a  comprehensive,  multi-faceted  account  of  its  effectiveness.  The  variety  of 
measurement  instruments  is  shown  in  Table  1 . 

DRC  and  RIH  developed  all  the  subjective  measurement  instruments  with  the  exception  of 
the  Maslach  Burnout  Inventory  and  the  RIH  patient  satisfaction  survey  which  are  commercially 
available  instruments.  With  respect  to  patient  satisfaction,  all  other  validation  hospitals  use  their 
own  locally-developed  surveys.  For  the  entire  measurement  suite,  the  goal  was  to  create 
instruments  that  could  be  used  in  a  wide  variety  of  hospitals. 

Objective  performance  data  consists  of  length  of  stay  data,  times  associated  with  selected 
treatment  procedures  such  as  thrombolytic  therapy,  times  recorded  in  medical  records  at  which 
significant  patient  assessment  and  treatment  activities  occurred,  in-patient  unit  assessment  of 
patient  preparation  for  admission  while  in  the  ED,  and  data  obtained  from  quality  assurance  (QA) 
reports,  personnel  reports,  and  the  trauma  registry.  Attitude  and  opinion  information  consists  of 
the  Maslach  Burnout  Survey,  ED  staff  members’  satisfaction  with  their  working  environment  and 
their  attitudes  towards  teamwork.  Patient  satisfaction  is  assessed  with  a  locally-developed  or,  as  is 
the  case  at  RIH,  a  commercially  available  patient  satisfaction  survey.  Teamwork  behavior  is 
assessed  with  the  BARS,  subjective  workload  assessment,  a  stress  scale,  the  Teamwork  Nursing 
Assessment,  and  distance  walked  per  shift  (pedometer  studies).  Program  evaluation  consists  of  a 
student  critique  and  course  evaluation  form  presented  to  ETCC™  students  at  the  completion  of 
their  classroom  training. 
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Table  1.  Evaluation  Measures  and  Instruments 


Measurement  Area 

Measures  and  Instruments 

ED  Performance 

•  ED  operations  (e  g.,  length  of  stay,  timed 
treatment  procedures) 

•  Admission  Evaluation  Survey 

•  Q  A  indicators 

•  Administrative  measures  (e.g.,  absenteeism) 

•  Trauma  registry  data  elements 

Attitude  and  Opinion 

•  Maslach  Burnout  Inventory 

•  Staff  attitudes  towards  teamwork 

•  Staff  job  satisfaction 

•  Patient  satisfaction 

•  Patient  wellbeing 

Team  Behavior 

•  Behaviorally  anchored  ratings  scales  (BARS) 
measuring  5  Team  Dimensions 

•  Observed  Error  Record 

•  Subjective  workload 

•  Stress  scale 

•  Teamwork  Nursing  Assessment 

•  Pedometer  studies 

Program  Evaluation 

•  ETCC™  course  critique 

Development  Activities 

ED  operations  at  RIH  were  examined  with  respect  to  data  normally  collected  for 
administrative  and  quality  assurance  purposes.  Examples  of  these  types  of  measures  are  length  of 
patient  stay  and  number  of  patients  who  sign  out  against  medical  advice.  Other  measures  were 
developed  based  on  key  concepts  of  the  ETCC™  curriculum.  Examples  include  assessment  of 
subjective  workload  and  degree  to  which  physicians  inform  nurses  that  a  patient  will  be  admitted 
to  the  hospital.  Operational  data  required  an  examination  of  the  source  and  details  of  the 
particular  information  collected.  No  data  collection  instrument  was  required  to  collect  these  data, 
but  a  process  for  isolating  the  relevant  data  elements  was  established.  Instruments  based  on  the 
ETCC™  curriculum  required  a  conceptualization  of  the  process  to  be  measured,  and  either 
identification  of  an  existing  instrument  or  creation  of  an  instrument  to  collect  the  data.  Once  a 
data  collection  approach  or  instrument  had  been  developed,  the  data  collection  process  was  begun 
to  examine  its  operational  characteristics.  These  characteristics  included  the  efficiency  of  data 
collection  and  user  acceptance  and  accuracy  in  completing  specially-designed  instruments.  Some 
instruments  underwent  or  are  undergoing  test-revise-retest  cycles  to  arrive  at  their  final  form. 

During  the  instrument  development  cycle,  data  were  gathered  on  the  amount  of  time  it 
took  a  respondent  to  complete  an  instrument  and  the  amount  of  time  required  to  score  or  record 
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information  from  that  instrument  or  an  operational  data  source.  This  information  was  added  to  a 
data  specification  sheet  that  was  created  for  each  kind  of  operational  datum  or  instrument. 

The  following  section  provides  the  current  versions  of  the  measurement  instruments  and 
the  data  specification  sheets  associated  with  selected  instruments.  Data  specification  sheets  will  be 
developed  for  all  measures.  Data  specification  sheets  in  this  report  are  developmental  drafts;  they 
will  be  standardized  and  contain  descriptive  and  implementation  information  helpful  to  validation 
site  users.  The  instruments  themselves  and  the  data  specification  sheets  will  provide  the  basis  for  a 
measurement  reference  manual  that  DRC  and  RIH  will  develop  for  use  by  the  validation  sites. 

This  manual  will  contain  hardcopy  and  electronic  versions  of  each  instrument,  detailed 
instructions  on  instrument  administration  and  scoring,  and  other  guidance  on  the  measurement 
activities  required  for  the  course  validation  effort. 
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Measurement  Instruments 


This  section  contains  all  instruments  developed  to  date  together  with  specifications  for 
these  instruments  and  others  still  being  pilot  tested.  The  instrument  is  introduced  by  a  title  page 
that  is  followed  by  the  instrument  and  the  data  specification  sheets  for  some  instruments.  Table 
presents  a  listing  of  the  measures  developed  and  their  location  in  this  report. 

Table  2.  Index  to  Evaluation  Measure  and  Instrument  Specifications 


ETCC™  Course  Evaluation 


5 


Length  of  Stay 


1.  Measure  Category:  ED  Performance 

2.  Type  of  Measure:  Operational 

3.  Status  of  Development:  Completed 

4.  Data  Elements:  Fixed 


5.  Data  Fields: 


a.  Date  patient  enters  ED  or  is  registered 

b.  Time  patient  enters  ED  or  is  registered 

c.  Date  patient  is  discharged  from  ED 

d.  Time  patient  is  discharged  from  ED 


6.  Comments: 


At  some  institutions  this  measure  is  collected  by  an  automated 
system.  For  those  institutions  without  automated  support,  a  sample 
of  medical  records  will  be  conducted  to  obtain  these  data  elements. 
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LENGTH  OF  STAY 


Data  Specifications 


Data  Item  Descriptor 

Definition  and  Specifications 

Notes 

Length  of  Stay  (LOS) 

Examines  ED  teamwork  as  it  related  to 
throughput  time:  Triage-Discharge. 
Retrospective  measure  designed  to 
determine  the  functional  relationship 
between  ETCC™  and  the  impact  upon 
LOS. 

Sampling  Scheme 

All  patients  admitted  and  discharged  from 
the  ED. 

Collection 

•  Resources  Needed:  QA  monitor. 

•  Measure:  QA  reporting  tool. 

•  Time  Estimation:  site  specific. 
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Throughput  Times 


1.  Measure  Category:  ED  Performance 

2.  Type  of  Measure:  Operational 

3.  Status  of  Development:  Completed 

4.  Data  Elements:  Fixed 

5.  Data  Fields: 

a.  Date  patient  enters  ED 

b.  Time  patient  enters  ED 

c.  Time  patient  is  registered 

d.  Time  of  triage 

e.  Time  patient  is  placed  in  treatment  area 

f.  Time  to  first  medication  or  procedure 

g.  Time  physician  sees  patient 

h.  Date  patient  is  discharged  from  ED 

i.  Time  patient  is  discharged  from  ED 

6.  Comments:  At  some  institutions  these  measures  are  collected  by  an  automated 

system.  For  those  institutions  without  automated  support,  a  sample 
of  medical  records  will  be  conducted  to  obtain  these  data  elements. 


Timed  Procedures 


1.  Measure  Category:  ED  Performance 

2.  Type  of  Measure:  Operational 

3.  Status  of  Development:  Completed 

4.  Data  Elements:  Variable 

5.  Data  Fields: 

a.  Thrombolytic  Therapy  for  Chest  Pain 

1)  Arrival  time  in  ED 

2)  TimetoEKG 

3)  EKG  to  Decision  for  tP A 

4)  Discharge  time 

b.  Thrombolytic  Therapy  for  Stroke 

1)  Arrival  time  in  ED 

2)  Time  to  CT 

3)  EKG  to  Decision  for  tPA 

4)  Discharge  time 

c.  Chest  Pain  -  Time  to  Catheterization  Laboratory 

1)  Arrival  time  in  ED 

2)  EKG  Time 

3)  Time  to  Catheterization  Lab 

4)  Time  patient  left  ED 

d.  Time  to  Charcoal  Treatment 

1)  Arrival  time  in  ED 

2)  Time  to  medication  or  treatment 

e.  Pediatric  Asthma  Pathway 

1)  Arrival  time  in  ED 

2)  Time  to  medication  or  treatment 
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f.  Time  to  Antibiotic  in  Meningitis  or  Septic  Children 

1)  Arrival  time  in  ED 

2)  Time  to  medication  or  treatment 

g.  Time  from  Medication  Order  of  Medication  Administration 

1)  Arrival  time  in  ED 

2)  Time  to  medication  or  treatment 

h.  Time  to  Antibiotic  in  Fever  in  Children  under  3  Months  of  Age 

1)  Arrival  time  in  ED 

2)  Time  to  medication  or  treatment 
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TIMED  PROCEDURES: 

THROMBOLYTIC  THERAPY  FOR  CHEST  PAIN 


Data  Specifications 


Data  Item  Descriptor 

Definition  and  Specifications 

Note 

Thrombolytics 

Objective  outcome  measure: 
Assess  for  ED  teamwork 
throughput  time:  door  to  drug. 

Sampling  Scheme 

All  ED  patients  receiving  tPA. 

Collection 

•  Resources  Needed:  QA 
monitor. 

•  Measure: 

-site  specific  QA  tool. 

•  Time  Estimation:  site 
specific 
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Quality  Assurance  Indicators 


1.  Measure  Category:  ED  Performance 

2.  Type  of  Measure:  Operational 

3.  Status  of  Development:  Completed 

4.  Data  Elements:  Variable 

5.  Data  Fields: 

a.  Number  of  patients  who  left  against  medical  advice  (AMA) 

b.  Number  of  patients  who  left  without  being  seen  (LWBS) 

c.  Number  of  patients  who  left  before  treatment  completed 

d.  Number  of  mislabeled  laboratory  specimens 

e.  Number  of  mislabeled  blood  tubes 

f.  Number  of  blood  culture  specimens  contaminated 

g.  Number  of  medication  errors 

h.  Number  of  patient  complaints 

i.  Number  of  patient  compliments 

j.  Number  of  unexpected  patient  returns  within  48  or  72  hours 

k.  Number  of  treatment  delays  due  to  consultants 

l.  Number  of  patient  falls 

m.  Number  of  ED  deaths 

6.  Comments:  Validation  sites  collect  one  or  more  of  these  elements,  but  not 

necessarily  all  of  them.  Frequency  of  reporting  may  be  weekly  or 
monthly. 
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QUALITY  ASSURANCE  INDICATORS 


Data  Specifications 


Data  Item  Descriptor 

Definition  and 
Specifications 

Note 

Quality  Assurance  Data 

Longitudinal  adverse  medical 
event  report;  retrospective 
measure  designed  to  determine 
the  functional  relationship 
between  MedTeams  (training) 
Emergency  Team 

Coordination  Course  and 
adverse  medical  events. 

Sampling  Scheme 

All  patients  admitted  to  ED 
currently  have  their  care 
reviewed  against  the 
standards. 

Location  for  Data 
Components 

Indicator/V  ariable: 

-  LWBS  -  not  registered  - 
ER01 

SMS  Report. 

-  AMA  -  registered  ER01 
SMS  Report. 

-  Unscheduled  returns  (72 
hours) 

PMS-5NPA  -  Davol  IS 
report. 

PMS-5NPC  -  Hasbro  IS 
report. 

-  Complaints  -  falls,  MED 
errors, 

incident  reports. 

-  Pt.  in  ED  >  6  hrs  -  SMS 
report. 

•  Know  where  to  find  the  data 
currently  gathered  by  your 
institution. 

•  Know  what  information  service 
report  it  is  retrieved  from  if  relevant. 
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Administrative  indicators 


1.  Measure  Category:  ED  Performance 

2.  Type  of  Measure:  Operational 

3.  Status  of  Development:  Completed 

4.  Data  Elements:  Variable 

5.  Data  Fields: 

a.  Hours  of  sick  leave  used 

b.  Number  of  incidents  of  tardiness 

c.  Number  of  unexcused  absences 

d.  Number  of  staff  resignations  or  terminations 

e.  Net  revenue  per  visit 

f.  Total  expense  per  visit 

g.  Number  of  visits  per  day 

h.  Number  of  admissions  to  inpatient  units 

i.  Census  per  hour 

j.  Acuity  level  of  ED  patients  per  day 

6.  Comments:  Frequency  of  measurement  of  these  indicators  varies  from  daily, 

weekly,  or  monthly. 
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Trauma  Registry  Indicators 


1.  Measure  Category:  ED  Performance 

2.  Type  of  Measure:  Operational 

3.  Status  of  Development:  Completed 

4.  Data  Elements:  Fixed  (National  TRACS  Data  Base) 


5.  Data  Fields: 

a*  Demographics 

1 )  11  indicators  that  identify  the  patient  are 
collected  at  sites  but  will  not  be  recorded 
in  the  validation  database 

2)  Patient  sex 

3)  Patient  date  of  birth 

b.  Injury 

1)  Date  of  injury 

2)  Time  of  injury 

3)  City  of  injuiy 

4)  State  of  injury 

5)  Zip  code  of  injury 

6)  Blunt  or  penetrating  injury 

7)  Cause  of  injury  (E-Code) 

8)  Site  of  injury 

9)  Police  report  number 

1 0)  Position  of  patient  in  vehicle 

11)  Safety  equipment  in  use 

c.  Prehospital 

1)  NameofEMS 

2)  EMS  run  number 

3)  Condition  of  patient  at  scene 

4)  Dispatch  date  of  EMS 

5)  Dispatch  time  of  EMS 

6)  Scene  arrival  time  of  EMS 

7)  Scene  departure  time  of  EMS 

8)  Arrival  time  at  first  hospital 

9)  Total  scene  time 

10)  Total  transport  time 

11)  Pulse  at  scene 

1 2)  Respiratory  rate  at  scene 

1 3)  Blood  pressure  at  scene 

14)  Eye  movement  at  scene 

15)  Verbal  response  at  scene 

16)  Motor  response  at  scene 

17)  Glasgow  Coma  Score  at  scene 


1 8)  Revised  Trauma  Score  at  scene 

19)  Use  of  CPR  at  scene 

20)  Airway  management  at  scene 

21)  Use  of  MAST  trousers  at  scene 

22)  Use  of  fluids  at  scene 

d.  Referring  Hospital 

1 )  Hospital  transfer 

2)  Referring  hospital  name 

3)  Referring  physician 

4)  Arrival  date  at  ref  hospital 

5)  Arrival  time  at  ref  hospital 

6)  Discharge  date  from  ref  hospital 

7)  Discharge  time  from  ref  hospital 

8)  Pulse  at  ref  hospital 

9)  Respiratory  rate  at  ref  hospital 

1 0)  Blood  pressure  at  ref  hospital 

11)  Eye  movement  at  ref  hospital 

12)  Verbal  response  at  ref  hospital 

13)  Motor  response  at  ref  hospital 

14)  Glasgow  Coma  Score  at  ref  hospital 

1 5)  Revised  Trauma  Score  at  ref  hospital 

e*  Emergency  Department  Admission 

1 )  Mode  of  transport 

2)  Arrive  from 

3)  Direct  admission 

4)  Arrival  date  in  ED 

5)  Arrival  time  in  ED 

6)  Discharge  time  from  ED 

7)  Chief  complaint 

8)  Arrival  condition 

9)  Trauma  surgeon  response  time 

10)  Neurosurgeon  response  time 

1 1 )  Temperature  at  ED 

12)  Pulse  in  ED 

13)  Respiratory  rate  in  ED 

14)  Blood  pressure  in  ED 
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15)  Eye  movement  in  ED 

16)  Verbal  response  in  ED 

17)  Motor  response  in  ED 

1 8)  Glasgow  Coma  Scale  on  ED  arrival 

19)  Revised  Trauma  Score  on  ED  arrival 

f.  Emergency  Department  Treatment 

1)  Head  CT  scan  in  ED 

2)  Abdominal  CT  scan  in  ED 

3)  Peritoneal  lavage  m  ED 

4)  Arteriogram  in  ED 

5)  Airway  management  in  ED 

6)  Use  of  CPR  in  ED 

7)  Units  of  blood  given  in  ED 

8)  Drug  screen  results  in  ED 

9)  Blood  alcohol  level  in  ED 

10)  Hematocrit  in  ED 

11)  Base  deficit  in  ED 

12)  ED  disposition 

13)  Admit  services 

g.  Hospital  Diagnoses 

1)  ICD-9  diagnosis  codes  1  through  10 
(uses  10  data  fields) 

2)  AIS  score  of  diagnoses  1  through  10 
(uses  10  data  fields) 

3)  Automatic  calculation  for  AIS  Score  for: 
head  and  neck,  face,  chest,  abdomen, 
extremities,  skin  and  soft  tissue 

4)  Automatic  calculation  for  ISS 

h.  Operations 

1)  Operation  ICD-9  for  codes  1  through  10 
(uses  10  data  fields) 

2)  ICD-9  description  codes  1  through  10 
(uses  10  data  fields) 

3)  Date  of  operation  for  codes  1  through  10 
(uses  10  data  fields) 

4)  Time  of  operation  for  codes  1  through  1 0 
(uses  1 0  data  fields) 

l  QA  Indicators 

1 )  Scene  time  greater  than  20  minutes 

2)  No  EMS  report 

3)  GCS  less  than  1 3  and  no  head  CT 

4)  No  neurological  documents 

5)  No  hourly  chart  documents 

6)  Uncontrolled  airway  in  comatose  patient 

7)  Patient  readmitted  within  72  hours  after 
initial  evaluation 

8)  Nonoperative  abdominal  GSW 

9)  Late  craniotomy  for  intracranial 
hemorrhage 

10)  Late  laparotomy  (>2  hours) 


1 1)  Interhospital  transfer  (>6  hours) 

1 2)  Late  treatment  of  open  tibial  fracture  (>8 
hours) 

13)  Unplanned  reoperation  within  48  hours 

14)  Late  truncal,  cranial  or  vascular  operation 
(>  24  hours) 

15)  Nonsurgeon  admission 

16)  Nonoperative  fixation  of  femoral  shaft 
fracture 

1 7)  Cervical  spine  fracture  missed  on 
admission 

j.  Complications 

1 )  Deep  venous  thrombosis 

2)  Hemo/pneux  thorax 

3)  Shock 

4)  Empyema 

5)  Cardiac  arrest 

6)  Intra-abdominal  abscess 

7)  Myocardial  infarction 

8)  Other  abscess 

9)  Congestive  heart  failure 

10)  Sepsis 

11)  Coagulopathy 

12)  GI  bleed 

13)  Compartment  syndrome 

14)  Pseudomemb.  Colitis 

15)  Arrythmia 

16)  Small  bowel  obstruction 

17)  Arterial  occlusion 

18)  Ent.  Cutan  fistula 

19)  Abd  wound  complication 

20)  Enterotomy 

21)  Decubiti 

22)  Cholecystitis 

23)  Acute  renal  failure 

24)  Hyperbilirubinemia 

25)  UTI 

26)  Stroke 

27)  Respiratoryfailure 

28)  Encephalopathy 

k*  Outcome 

1 )  Self  care 

2)  Mobility 

3)  Verbal 

4)  FIM  score 

5)  Days  in  the  ICU 

6)  Days  in  the  hospital 

7)  Hospital  charges 

8)  Reimbursed  charges 

9)  Hospital  disposition 

10)  Data  entry  complete 
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Admission  Evaluation  Survey 


1.  Measure  Category:  ED  Performance 

2.  Type  of  Measure:  Survey 

3.  Status  of  Development:  Completed 

4.  Data  Elements:  Fixed 

Comments  text 

5.  Data  Fields:  Form 
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•  You  have  just  received  a  patient  admitted  through  the  ED. 

•  In  support  of  MEDTEAMS,  an  ED  Research  project,  please  take  a  moment  to  provide  the  ED  with  process 
feedback. 

•  This  form  is  self  explanatory  and  meant  to  be  completely  anonymous.  Please  do  not  write  any  caregiver  or 
patient  names  on  the  form  or  the  information  must  be  discarded. 

•  When  the  form  is  completed  place  in  the  specially  provided  envelope  on  your  unit  labeled  “Mary  Salisbury 
RN*.  Your  Clinical  Manager  will  return  the  completed  forms  on  a  regular  basis. 

Thank  you  for  your  time;  although  voluntary,  your  participation  is  very  supportive. 


1. 

Time  elapsed  between  report  and  patient  arrival  to  patient  unit. 

hrs _ 

_min 

2. 

Is  the  patient  in  pain? 

Yes 

No 

If  the  answer  is  YES,  does  the  patient  know 
what  to  do  with  respect  to  pain? 

Examples:  Call  for  the  nurse 

Yes 

No 

Is  aware  pain  Rx  is  ordered  every  4  hours 

3. 

Have  all  ordered  ED  medications  been  given? 

Yes 

No 

If  the  answer  is  NO,  were  you  informed  why  not? 

Yes 

No 

4. 

Are  intravenous  line  and  medications  running  at  their 
described  rate? 

Yes 

No 

NA 

5. 

Is  the  patient  aware  of  why  he  or  she  was  admitted? 

Yes 

No 

See  Comments 

6. 

Were  family  members  or  guardians  advised  of  the  patient’s  condition? 

Yes 

No 

See  Comments 

7. 

Are  family  members  or  guardians  aware  that  the  patient  was  admitted  to 
your  unit? 

Yes 

No 

See  Comments 

8. 

Is  the  patient’s  condition  consistent  with  the  ED  report? 

Yes 

No 

9. 

Has  the  patient  been  provided  with  an  identification  bracelet? 

Yes 

No 

10. 

Was  report  given  by  that  patient’s  primary  nurse? 

Yes 

No 

11. 

Was  the  ED  nurse’s  report  professional? 

Yes 

No 

Examples  of  professional  expectations: 

(1)  A  clear  and  well  communicated  systems  report 

(2)  The  ED  nurse  identifies  self  by  name 

(3)  The  ED  nurse  encourages  ucall  back”  from  receiving  RN 
if  there  are  any  questions  or  concerns . 

Comment s: 
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ADMISSION  EVALUATION  SURVEY 


Data  Specifications 


Data  Item  Descriptor 

Definition  and  Specifications 

Note 

Admission  Evaluation 
Survey 

•  Outcome  oriented 
dichotomous  assessment 
measure:  Designed  to 
measure  ED  team 
achievement  of  specific 
standard  based  outcomes. 

•  Dichotomous  type 
assessment  chosen  to  gather 
factual  information. 

•  Closed  question  efficiency 
increases  predictability  of 
time-to-completion. 

•  Envisioned  as  a  “high 
risk”  questionnaire 
from  the  ED 
practitioners’ 
perspective. 

•  Staff  received 
education  specific  to 
the  tool  and  the  intent 
of  the  tool. 

all  info  confidential 

•  all  info  reported  in  the 
aggregate 

Sampling  Scheme 

•  Placed  on  admission 
record/chart  for  all  admitted 
patients. 

•  All  data  accounted  for  by 
numbering  assessment 
forms. 

Admission  clerk  function. 

Collection  Time  Estimate 

•  Resources  Needed: 

-Completed  by  “in- 
house”  nursing  staff. 

-Scored  by  research 
assistant. 

-Data  entered  by 
research  assistant. 

-Analyzed  by  PI 

•  Time  Estimation.  1  minute 
average  per  respondent,  t 
time  to  complicated  medical 
plans  of  care. 
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Maslach  Burnout  Inventory 


1.  Measure  Category:  Attitude  and  Opinion 

2.  Type  of  Measure:  Survey 

3.  Status  of  Development:  Completed 

4.  Data  Elements:  Fixed 

5.  Data  Fields:  Form 
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Rhode  Island  Hospital 

593  Eddy  Street 
Providence,  R.I.  02903 


_ N/A _ 

RIH  Project  #  Name  of  Patient 

AGREEMENT  TO  PARTICIPATE  IN  CLINICAL  INVESTIGATION 


Occupational  Stress  Within  the  Medical  Profession: 
An  Assessment  of  Burnout  and  Coping  Methods 


You  are  being  asked  to  participate  in  a  research  project  as  described  in  this  form 
below.  All  such  research  projects  carried  out  in  this  Hospital  are  covered  by  the 
rules  of  both  the  Federal  Government  and  the  Rhode  Island  Hospital.  These  rules 
require  that  you  give  your  signed  agreement  to  participate  in  this  project. 

The  co-investigators  will  explain  to  you  in  detail  the  purpose  of  the  project,  the 
procedures  to  be  used,  and  the  potential  benefits  and  possible  risks  of  participation. 
You  may  ask  him/her  any  questions  you  may  have  to  help  you  understand  the 
project.  A  basic  explanation  of  the  project  is  written  below.  Please  read  this 
explanation  and  discuss  any  questions  you  might  have  with  the  co-investigators. 

If  you  then  decide  to  participate  in  the  project,  please  sign  this  form  on  the  line 
below  in  the  presence  of  a  witness  and  the  person  who  explained  the  project  to  you. 
You  should  be  given  a  copy  of  this  form  to  keep. 


1.  Nature  and  Purpose  of  the  Project 

The  purpose  of  this  project  is  to  determine  both  the  level  of  burnout 
and  coping  mechanisms  among  Rhode  Island  Hospital  health  care 
workers.  Bumout  is  a  condition  that  can  seriously  impair  the 
performance  of  health  care  workers;  it  often  causes  stress, 
depersonalization  and  reduced  productivity  in  individuals.  Various 
studies  have  been  performed  examining  the  correlation  between 
bumout  and  gender,  intradepartmental  workers,  and  prehospital 
personnel.  In  this  study,  however,  we  hope  to  assess  the  level  of 
bumout  in  Rhode  Island  Hospital  medical  professionals,  in  particular 
emergency  physicians,  surgeons,  and  internal  medicine  physicians. 
We  will  compare  the  levels  of  bumout  both  among  three  departments 
and  also  among  prehospital  personnel,  secretaries,  nurses,  medical 
students,  residents,  and  attending  physicians  within  these 
departments.  Furthermore,  we  would  like  to  evaluate  the  coping 
mechanisms  of  those  surveyed. 
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2. 


Explanation  of  Procedures 


Physicians  from  the  above  three  departments  will  be  asked  to 
voluntarily  complete  a  survey  to  determine  respondent 
demographics,  burnout  levels,  and  coping  mechanisms.  The  portions 
of  this  survey  designed  to  assess  bumout  and  coping  methods  are 
modeled  after  the  Malsach  Bumout  Inventory  ( 1993),  a  standard 
model  used  to  assess  these  parameters. 

Any  questions  regarding  the  survey  should  be  directed  to  Dr.  Bruce 
Becker  through  any  of  the  three  co-investigators,  Dinesh  Bahl 
(401)863-4858,  Eric  Blackiston  (401)  863-6428,  and  Carl  Chang 
(401)  863-4858.  Dr.  Becker  may  be  contacted  at  (401)  444-541 1 
(Rhode  Island  Hospital  Emergency  Department). 

3.  Discomforts  and  Risks 

We  do  not  foresee  any  risks  or  discomforts  by  participating  in  this 
survey. 

4.  Benefits 

We  hope  that  we  will  both  learn  much  about  physician  bumout  from 
this  study  and  that  it  will  eventually  result  in  changes  in  policy  that 
may  lead  to  alterations  in  your  work  environment. 

5.  Alternative  Therapies 
Non-applicable. 

6.  Confidentiality 

All  records  relating  to  this  project  will  be  handled  and  safeguarded 
according  to  standard  Hospital  policy  for  all  medical  records.  Y our 
medical  record  will  always  be  handled  in  conformity  with  the  Rhode 
Island  Act  relating  to  the  confidentiality  of  health  care  information. 

The  information  provided  in  this  survey  will  be  kept  confidential. 
The  principal  investigator  will  not  have  access  to  the  completed 
survey  or  database  used  in  this  study.  When  the  results  are  reported 
there  will  be  no  way  for  readers  to  single  out  individual  responses. 
Furthermore,  when  data  is  reported,  no  correlation  will  be  made 
between  demographic  parameters  and  individual  items  in  the  survey; 
however,  data  from  the  survey  will  be  correlated  with  the  three 
subscales. 

7.  Refusal/Withdrawal 


Participation  in  this  project  is  completely  voluntary. 
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8. 


Information 


We  do  not  expect  any  (further)  unusual  risks  as  a  direct  result  of  this 
project.  However,  should  an  unforeseen  physical  injury  occur, 
appropriate  medical  care,  as  determined  by  the  Hospital,  will  be 
provided  but  no  financial  compensation  will  be  given.  Further 
information  in  regard  to  this  provision  can  be  obtained  by  calling  the 
Office  of  Research  Administration  at  444-6246. 


If  you  are  experiencing  difficulties  of  any  type,  please  feel  free  to 
call  the  appropriate  resource(s). 

RIH  EAP  Program  (401)  732-9444 

(800)445-1195 

-  counseling  for  any  issues  in  an  environment  of  strict  confidentiality. 

Alcoholics  Anonymous  (401)  438-8860 

Cocaine  Anonymous  (401)  276-3039 

Narcotics  Anonymous  (401)  461-1110 
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Appendix  A 


I  ACKNOWLEDGE  THAT  I  HAVE  READ  AND  FULLY  UNDERSTAND  THE 
ABOVE  CONSENT.  THAT  ALL  OF  MY  QUESTIONS  HAVE  BEEN 
SATISFACTORILY  ANSWERED.  AND  I  AGREE  TO  PARTICIPATE  IN  THIS 
PROJECT. 

□  YES  □  NO 

The  purpose  of  this  survey  is  to  discover  how  medical  professionals  view  their  job  and 
the  people  with  whom  they  work  closely. 

Age:  _ 

Gender  _ 

Marital  Status:  _ 

Ethnicity:  _ 

Position  and  Year  (if  applicable):  _ 

example:  3rd-year  medical  student 

Hours  worked  per  week:  _ 

Number  of  hours  per  shift:  _ 

Please  indicate  the  shift  worked  most  often: 

7  AM -3:15  PM  □  11:15  PM-7:15AM 

3  PM  -  1 1 :30  PM  Q  Other: _ 

Number  of  years  in  particular  profession:  _ 

Number  of  years  you  expect  to  remain  in 

your  current  profession:  _ 


□ 

□ 
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Below  are  41  statements  of  job-related  feelings.  Please  read  each  statement  carefully  and 
decide  if  you  ever  feel  this  way  about  your  profession.  If  you  have  never  had  this  feeling, 
write  a  "0"  (zero)  in  the  space  before  the  statement.  If  you  have  had  this  feeling,  indicate 
how  often  you  feel  it  by  writing  the  number  (from  1  to  6)  that  best  describes  how 
frequently  you  feel  that  way.  Below  is  an  example: 


Example: 


HOW  OFTEN: 

0  1  2 

Never  A  few  Once  a 

ti  mes  a  year  month  or 

or  less  less 

3  4 

A  few  Once  a 

times  a  week 

month 

5 

A  few 

times  a  week 

6 

Everyday 

HOW  OFTEN 
0-6 

Statement: 

1  feel  depressed  at  work. 

If  you  never  feel  depressed  at  work,  would  write  the  number  "0"  (zero)  under  the  heading 
"HOW  OFTEN."  If  you  rarely  feel  depressed  at  work  (a  few  times  a  year  or  less),  you 
would  write  the  number  "1".  If  your  feelings  of  depression  are  fairly  frequent  (a  few 
times  a  week,  but  not  daily)  you  would  write  a  "5." 

HOW  OFTEN: 

0  1  2 

Never  A  few  Once  a 

times  a  year  month  or 
or  less  less 

3  4 

A  few  Once  a 

times  a  week 

month 

5 

A  few 

times  a  week 

6 

Everyday 

HOW  OFTEN 
0-6 

Statements: 

1. 

I  feel  emotionally  drained  from  my  work. 

2. 

I  feel  used  up  at  the  end  of  the  workday. 

3.  _  I  feel  fatigued  when  I  get  up  in  the  morning  and  have  to  face 


4. 

5. 

6. 

7. 

8. 

9. 

10. 
11. 
12. 


another  day  on  the  job. 

I  can  easily  understand  how  my  patients  feel  about  things. 

I  feel  I  treat  some  patients  as  if  they  were  impersonal  objects. 
Working  with  staff  and  patients  all  day  is  really  a  strain  for  me. 

1  deal  very  effectively  with  the  problems  of  my  patients. 

I  feel  burned  out  from  my  work. 

I  feel  I'm  positively  influencing  other  people's  lives  through  my  work. 
I've  become  more  callous  toward  patients  since  I  took  this  job. 

I  worry  that  this  job  is  hardening  me  emotionally. 

I  feel  very  energetic. 
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13. 

14. 

15. 

16. 

17. 

18. 


19. 

20. 
21. 
22. 


I  feel  frustrated  by  my  job. 

I  feel  I'm  working  too  hard  on  my  job. 

I  don't  really  care  what  happens  to  some  patients. 

Working  with  staff  and  patients  directly  puts  much  less  stress  on  me. 
I  can  easily  create  a  relaxed  atmosphere  with  my  patients. 

I  feel  exhilarated  after  working  closely  with  my  patients. 

I  have  accomplished  many  worthwhile  things  in  this  job. 

I  feel  like  I'm  at  the  end  of  my  rope. 

In  my  work,  I  deal  with  emotional  problems  very  calmly. 

I  feel  patients  blame  me  for  some  of  their  problems. 


Thank  you  for  completing  this  survey! 

Please  return  this  survey  to  either  Dinesh,  Eric,  or  Carl,  or  deposit  it  in 

in  the  ACM  Room. 
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MASLACH  BURNOUT  INVENTORY 


Data  Specifications 


Data  Item 
Descriptor 

Definition  and  Specifications 

Notes 

Maslach  Burnout 
Inventory 

•  Self  report  rating  survey  used  to  determine 
the  degree  of  burnout  for  hospital  ED 
personnel. 

•  Rating  Statements:  Respondents  asked  to 
judge  statement  along  an  ordered  dimension. 

•  Maslach  &  Jackson  (1986)*  define  emotional 
exhaustion  as  being  “emotionally 
overextended  and  exhausted  by  one’s  work.” 
Depersonalization  is  described  as  having  “an 
unfeeling  and  impersonal  response  towards 
recipients  of  one’s  care  of  service,”  and 
personal  accomplishment  is  characterized  as 
having  feelings  of  “competence  and 
successful  achievement  in  one’s  work  with 
people”. 

Sampling  Scheme 

•  Target  population:  All  ED  RNs,  Drs.  and 
technicians 

•  All  data  accounted  for  by  numbered  surveys. 

Collection 

•  Resources  needed:  research  assistant. 

•  Measure: 

-administered  by  research  assistant. 

-scored  by  research  assistant. 

-Data  entered  by  research  assistant. 
-Analyzed  by  PI. 

•  Time  Estimation  -  15  min/respondent. 

& 

Maslach,  C.,  and  S.E.  Jackson.  (  1986).  Maslach  Burnout  Inventory  Manual.  2nd  Ed.  Palo 

Alto:  Consulting  Psychologist  Press. 
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Staff  Attitude  and  Job  Satisfaction  Survey 


1.  Measure  Category:  Attitude  and  Opinion 

2.  Type  of  Measure:  Survey 

3.  Status  of  Development:  Under  Development 

4.  Data  Elements:  Fixed 

5.  Data  Fields:  Form 
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Patient  Satisfaction  Survey 


1.  Measure  Category: 

2.  Type  of  Measure: 

3.  Status  of  Development: 

4.  Data  Elements: 

5.  Data  Fields: 

6.  Comments: 


Attitude  and  Opinion 
Survey 

Review  and  modification  of  hospital-specific  instruments 

Fixed 

Form 

Hospitals  use  a  set  of  questions  and  a  survey  format  (either  printed 
form  or  telephone  interview)  either  developed  locally  or  obtained 
from  a  commercial  vendor.  Pages  30  to  33  provide  a  copy  of  the 
instrument  used  at  RIH.  Page  35  provides  a  form  used  at  a  military 
hospital. 

Questions  specific  to  teamwork  processes  or  outcomes  are  being 
investigated  as  additions  to  existing  patient  satisfaction  surveys. 
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Emergency  Department 

Patient  Experience 

A  SM 

Survey 


Our  Commitment  to  Quality  Patient  Care 


Rhode  Island 
Hospital 


Please  help  us  to  improve  the  emergency  services  at  Rhode  Island  Hospital.  Listed  below  are 
experiences  patients  sometimes  encounter.  We  sincerely  hope  that  your  most  recent  visit  went 
smoothly.  If,  however,  it  didn't,  we  want  to  know. 


In  each  of  the  following  sections,  please  check  the  appropriate  box  G  to  indicate  whether  any 
item  was  experienced  during  your  vhrit.  If  the  question  does  not  apply  to  your  experience,  please 
leave  it  blank.  Then,  check  the  box  B  under  the  grade  that  represents  your  evaluation  of  hospital 
|i,i(<nnniiw  in  that  category.  Space  is  provided  at  the  end  of  the  questionnaire  for  any  comments 
yon  may  wish  to  make. 


_  1-4 
5*1 


PARKING  AND  ACCESS 


(Bye 


Please  indicate  B  your  experience  in  the  following  areas: 


Yes 

□ 

□ 

□ 

□ 

□ 

□ 

□ 


No 
□  *> 
□  « 
On 
On 
On 
On 
On 


Emergency  Department  p*T^T,g  area  was  easy  to  find. 

You  easily  found  a  parking  space. 

The  designated  parking  area  was  convenient  to  the  Emergency  Department. 

A  wheelchair  was  available,  if  needed. 

You  had  to  pay  for  parking. 

ThcLporking  area  was  well  lighted. 

Assistance'from-staff  in  pvting  yon  out  of  your  vehicle  was  provided,  if  needed. 


Overall,  your  parking  and  access  experience  was: 

Excellent  Good  Fair 

A  A-  B+  B  B-  C  + 

□  O  □  □  D  □ 


10 


07 


05 


C 

o 


Poor  Unacceptable 
C-.0  +  D  F 

Q  □  □  □ 


02 


01 


00 


29-30 


TRIAGE  PROCEDURES 
Did  a  nurse  (we  call  this  person  a  "triage  nurse”)  see  you  at  the 
front  desk  prior  to  being  registered? 

Please  indicate  B  your  experience  in  the  following  areas: 

!*o 

M  Your  physical  condition  was  adequately  assessed  by  the  triage  nurse, 
ss  Triage  nurse  treated  you  kindly. 

36  Privacy  during  triage  was  adequate. 

Sr  The  triage  area  was  adequately  staffed  for  the  number  of  patients. 

Overall,  triage  procedures  were: 

Excellent  Good  Fair  Poor 

6  □  Bd  fi  □  <d  fi  6  Dd  fi 


CD  i  Yes  OiNo  33 


Unacceptable 

fi 


30-39 


30 


6398 


Please  indicate  0  your  experience  in  the  following  areas: 

Yes  No 

|  |  |  1 4Q  Registration  procedures  were  clear. 

□  You  were  treated  kindly  by  the  staff  during  registration. 

□  0«  The  number  of  forms  required  for  registration  was  reasonable. 

□  □«  Privacy  during  registration  was  adequate. 

n  0  44  Time  spent  on  registration  procedures  was  appropriate. 

0  □  45  Adequate  information  about  payment  or  billing  was  provided. 

■  0  You  were  informed  of  reason  for  any  delay  in  registration. 

0  047  The  reception  area  was  adequately  staffed  for  the  number  of  patien  . 

Overall,  registration  procedures  were.  Unacceptable 

Excellent  Good  c  C-  D  +  D  F 

6  □  □  fi  □  □  fi  D  □  0  □ 

1 — 1  “  no  07  06  05 


10 


09 


03 


01 


01 


00 


48-49 


WAITING  ROOM 

Please  indicate  0  your  experience  in  the  following  areas: 


O  0  si  Waiting  room  was  quiet. 

O  0  sj  Waiting  room  was  dean. 
n  O  54  Waiting  room  was  comfortable. 

O  Oss  Waiting  room  seating  capacity  was  adequate. 

O  Oss  You  were  informed  of  reason  for  any  delay. 

Overall,  the  waiting  room  was: 

Excellent  Good 

A  A-  B+  B  B-  C  + 

a  □  □  □  o  □ 


No 

0  57  Television  was  available. 

Osa  Length  of  waiting  time  was  reasonable. 
059  The  restroom  was  easy  to  find. 

□  «  The  restroom  was  dean. 

Oti  Refreshment  facilities  were  available. 

Poor  Unacceptable 

C-  D+  D  F 

n  n  o  O 
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TREATMENT 


indicate  0 

Yes 

No 

O 

068 

o 

069 

o 

070 

o 

On 

o 

On 

o 

073 

0 

074 

o 

075 

iuiv. 

[  was  kept  informed  about  your  condition. 


ipngrn  ui  wau  ui  aw,  •  - 1 

Your  condition  was  correctly  diagnosed  and  treated. 


Nurses 


Appropriate  time  was  spent  in  identifying  problem  by: 
Staff  introduced  themsehres  and  identified  their  role: 

/y  fvp1«i«tinn  about  condition  ct  treatment  was  giv 

You  were  treated  kindly  by: 

All  of  your  questions  were  answered  by: 

Your  problem  was  taken  seriously  by- 
Your  privacy  was  respected  by. 


Emergency 

Physicians 

l 

2 

Yes 

No 

O 

O 

0 

O 

O 

0 

0 

O 

0 

0 

O 

0 

O 

0 

Specialist 

Physicians 

1 

2 

Yes 

No 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

'technicians 

1 

2 

Yes 

No 

Not  Applicable 

O 

0 

Not  Applicable 

0 

0 

0 

0 

O 

0 

O 

0 

13-16 


33-41 

41-45 


Overall,  treatment  was: 

Excellent  Good 

A  A-  B+  B 

□  □  □  □ 


Fair 

B-  C+  C 

□  0  □ 


Poor  Unacceptable 

C-  D+  D  F 

0  0  □  □ 
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X-RAY  AND  LABORATORY  SERVICES 

(If  you  did  not  receive  X-«y  or  laboratory  *TTice.,  plea*  .kip  tbi»  *“«»“ 

Please  indicate  0  your  experience  in  the  following  areas: 


X-ray 

Services 


Lab 

Tests 


Transportation  to  X-ray  area  was  satisfactory. 

Length  of  wait  for  X-ray  or  laboratory  tests  was  reasonable. 
You  were  treated  with  concern  by  staff. 

Privacy  during  X-ray  was  adequate. 

Staff  provided  a  satisfactory  explanation  of  test  procedures. 

An  attendant  was  present  during  X-ray  services. 

Length  of  wait  for  results  of  X-ray  or  lab  tests  was  reasonable. 
Privacy  and  comfort  for  specimen  collection  was  satisfactory. 
Staff  demonstrated  adequate  skill  in  performing  tests. 


Yes  No 

□  □ 

□  H 

□  H 

□  □ 

□  □ 

□  H 

□  □ 
Not  Applicable 

□  □ 


10 


09 


.IVe- 

Good 

Fair 

Poor 

B  + 

□ 

08 

B 

□ 

07 

B- 

□ 

06 

C  + 

□ 

05 

c 

□ 

04 

c- 

□ 

03 

D  + 

□ 

01 

were: 

Poor 

Good 

Fair 

B  + 

□ 

B 

□ 

B- 

□ 

C  + 
□ 

c 

□ 

C- 

□ 

D  + 

□ 

D 


Yes  No 

Not  Applicable 

□  □ 

□  □ 
Not  Applicable 

□  □ 
Not  Applicable 

□  □ 
□  □ 
□  □ 


Unacceptable 

F 

□ 

oo 

Unacceptable 

F 

□ 


DISCHARGE  PROCEDURES 

(If  you  wen  admitted  to  the  hospital  through  the  Emctgency  Department,  pieaae  skip  this  tectum.) 

Please  indicate  0  your  experience  in  the  following  areas: 

Yes  No 

□  CU  Discharge  procedures  were  understandable. 

—  You  were  treated  with  courtesy  by  nursing  staff  during  discharge  procedures. 

After  physician  ordered  discharge,  your  wait  was  appropriate. 

Instructions  about  medications,  continued  care,  or  condition  were  adequate. 

A  wheelchair  was  available,  if  needed. 

Appropriate  assistance  was  provided  by  staff  upon  departure. 

Transportation  pick-up  area  was  convenient. 


Overall,  discharge  procedures  were: 

Excellent  Good 

A  A-  B+  B 

□  □  □  □ 


Fair 

B-  C+  C 

□  □  □ 


Poor  Unacceptable 

C-  D+  D  F 

□  □  □  □ 


OVERALL  SATISFACTION 

Overall,  the  care  provided  during  your  visit  at  this  hospital  was: 

Excellent  Good  5“  „  r  ST  D 

6  □  □□□□□□  □ 

How  do  you  compare  out  Emergency  Depattmmt  services  with  other  local  hospitals! 
□  .Better  □  >  About  the  same  □.  Worse  □.  Don't  know 


Unacceptable 

F 

□ 


How  willing  are  you  to: 

Use  our  emergency  services  again: 
Recommend  our  services  to  others: 
Be  hospitalized  at  our  hospital: 


Very 

Willing 


Somewhat 

Somewhat 

Very 

Willing 

Unwilling 

Unwilling 

4 

1 

□ 

o 

□ 

□ 

□ 

□ 

.  □ 

□ 

□ 

□ 

68-69 


70-71 


5=3 


19-30 


33-34 


35 


32 


□  a  No 

CD  i  No 

□  2  No 

□  2  No 

□  2  No 


□  1  Yes  □  2  No 


ADDITIONAL  INFORMATION 

Did  you  arrive  by  ambulance?  □  1  Yes  CL  No 

If  you  came  by  ambulance,  were  you  able  to  choose  the  hospital?  0 1  Yes  O  2  No 

Were  you  transferred  from  another  hospital  to  Rhode  Island  Hospital?  D 1  Yes  O  2  No 

Have  you  ever  used  a  hospital  Emergency  Department  before?  □  1  Yes  □  2  No 

If  yes,  have  you  used  this  Emergency  Department  before?  □  1  Yes  □  2  No 

If  yes,  how  many  times  in  the  last  12  months?  _  «< 

Do  you  have  a  Primary  Care  physician  for  routine  care?  □  1  Yes  D  2  No 

If  YES,  why  didn't  you  go  to  your  Primary  Care  Physician? 

□  54  Ho  appointments  available  CD  ss  Need  was  urgent  CD  S6  Physician  referred  me  CD  S7  Other 

If  there  were  a  walk-in  center  for  urgent  care  in  another  convenient  location  that  were 
open,  would  you  have  gone  there  instead  of  the  Emergency  Department?  CD  1  Yes  CD  2  No 

Please  indicate  the  reason(s)  for  selecting  this  hospital  for  your  emergency  care. 

=  66  Close  to  home  _  »  Insurance  reasons  CD  72  Ambulance  personnel 

=  <7  Referred  by  physician  _  70  Hospital  reputation  selected 

— I «  Recommendations  of  family/friend  CD  71  Previous  experience  CD  73  Other 

Have  you  or  a  family  member  used  any  of  the  following  services  at  this  hospital  in  the 
last  12  months? 

Outpatient  clinics/ ambulatory  surgery  10  CD  1  Yes  CD  2  No  Inpatient  treatment  21  CD  1  Yes  CD  2  No 
How  long  did  you  have  to  wait:  o-is  16-30  1/2-1  1-2  2-3  3-6 

minutes  minutes  hour  hours  hours  hours 

i  ^  <  -L,  6 

To  be  registered?  I  M  M  Q  J  P 

In  the  waiting  zoom?  ^  * 

In  the  treatment  room  to  see  the  nurse?  ^ 

In  the  treatment  zoom  to  see  the  physician?  „  _ 

Tb  get  blood  drawn  or  an  ECG?  —  —  Z  Z  Z  Z 

For  an  X-ray?  _  Z  ""  ""  "" 

For  laboratory  test  results?  IH  M  H  M  Zl  Z 

In  all,  how  much  time  did  you  spend  in 

the  Emergency  Department?  CD  CD  CD  CD  CD  CD 

What  day  of  the  week  was  this  visit? 

Bi  Monday  Qilhesday  CD 3  Wednesday  CD  4  Thursday 

5  Friday  lJ  6  Saturday  LJ  7  Sunday 

What  time  of  day  did  you  arrive  for  this  visit? 

□  ,  7  am  -  3  pm  □2  3  pm  -  11  pm  CD  3  11  pm  -  7  am 
Your  age:  CD  1  Under  18  Gi  18-24  CDs  25-40  CD 4  41-64  CDs  65  and  over 
Your  sex:  CDi  Male  CDi  Female 
iype(s)  of  Health  Insurance: 

Js4  None/Self-pay  □  57  Medicare  □<soHMO/PPO  □  63  Other 

=  ss  Blue  Cross/Blue  Shield  Js*  Medicaid/Welfare  □«  CHAMPUS 

_ls«  Commercial  CD  59  Worker's  Compensation  CD<a  General  Public  Assistance  (GPA) 


B3  Wednesday 
7  Sunday 


CD  4  Thursday 


•so  HMO/PPO  □  «  Other 

61  CHAMPUS 

62  General  Public  Assistance  (GPA) 


We  would  appreciate  any  additional  comments  or  suggestions,  favorable  or  unfavorable: 


Patient  Name/Address/ftlephone  (Optional} 


Please  place  the  questionnaire  in  the  envelope  provided  and  mail  it  as  soon  as  possible  to: 

GORDON  S.  BLACK  CORPORATION,  135  Coiporate  Woods,  Rochester,  NY  14623-1457 

©  GORDON  S.  BLACK  CORPORATION  1993 
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Printed  on 


PATIENT  SATISFACTION  SURVEY 
EXAMPLE:  GORDON  BLACK  PATIENT  EXPERIENCE  SURVEY 

Data  Specifications 


Data  Item  Descriptor 

Definition  and  Specifications 

Notes 

Gordon  Black  ED  Patient 
Experience  Survey 

Multi  Method;  outcome  oriented 
survey.  Respondent  is  queried 
regarding  ED  experience. 
Provides  an  ability  to  study 
the  relationship  between  ED 
patient  satisfaction  and  the 
variables  related  to  ED 
experience. 

Know  the  type  of  patient 

satisfaction  survey  currently 
used  at  your  institution. 

Sampling  Scheme 

Pre  and  post  MedTeams  Training 
to  all  patients  seen  and 
discharged  from  ED  within 
one  month  timeframe. 

Know  how  often  and  when  the 
survey  is  administered  at  your 
site. 

Collection 

•  Resources  -  TBD* 

•  Measure:  Gordon  Black  time 
to  administer-site  specific 

-scoring  -  TBD 
-analyzed  by  PI 
-data  entry 

•  Time  Estimation  -  TBD 

•  Know  who  administers  and 
scores  the  survey  for  your 
institution. 

•  Come  prepared  to  discuss  your 
ability  to  gain  access  to  patient 
satisfaction  scores.  We  would 
need  the  ability  to  report  data  in 
the  aggregate  to  demonstrate 
the  impact  of  MedTeams 
training  on  those  scores. 
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Emergency  Medicine  Department  Exit  Survey 


Please  take  a  few  seconds  to  help  us  serve  you  better! 


Age: _  Date:_ 

Sex:  M  /  F  Race: 


Time: _ AM  /  PM 

Status: _ AD  /  Ret.  /  Family  Member 


Please  circle  the  number  that  best  shows  how  you  feet 


’1 "  =  Strongly  Disagree 


"5"  =  Strongly  Agree 


1.  I  understood  what  my  Doctor  told  me  to  do  when  I  go  home. 

1  2  3  4  5 

2.  I  understood  what  my  Nurse  or  Corpsman  told  me  to  do  when  I  go  home. 

1  2  3  4  5 

3.  I  understand  the  AfterCare™  Instruction  Sheet(s). 

1  2  3  4  5 

4.  I  can  read  and  understand  the  Hand-Written  Instructions. 

1  2  3  4  5 

5.  I  understand  how  to  use  the  Medicines  I  was  prescribed. 

1  2  3  4  5 

I  was  not  prescribed  any  Medicine _ 

6.  I  understand  what  to  do  if  I  have  any  more  problems. 

1  2  3  4  5 

7.  I  was  treated  with  respect  during  check-in. 

1  2  3  4  5 

8.  I  was  treated  with  respect  in  the  ER 

1  2  3  4  5 

9.  The  ER  Staff  kept  me  informed  about  the  progress 
of  my  evaluation  while  I  was  in  the  ER 

1  2  3  4  5 

10.  I  was  satisfied  with  the  care  I  received  in  the  Emergency  Room  today. 

1  2  3  4  5 


Comments: 


Patient  Wellbeing  Survey 


1.  Measure  Category: 

2.  Type  of  Measure: 

3.  Status  of  Development: 

4.  Data  Elements: 

5.  Data  Fields: 

6.  Comment: 


Attitude  and  Opinion 
Survey 

Field-testing  underway 

Fixed 

Form 

Two  questions  will  be  asked  at  triage  and  discharge.  Written  and 
oral  methods  of  presentation  of  the  questions  are  being  pilot-test  at 
four  hospitals. 
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Patient  wellbeing  Survey 


BEFORE  TREATMENT 

1 .  Wellness  -  How  sick  are  you  feeling  on  a  scale  of  1  -  1 0,  with  1  being  not  very  sick  and  1 0 
being  very  sick. 

123456789  10 

Not  Very  Very  Sick 

Sick 

2.  Pain  -  How  bad  is  your  pain  on  a  scale  of  1  -  10,  with  1  being  no  pain  and  10  being  severe 
pain. 


123456789  10 

Not  much  Severe 

Pain  Pain 


AFTER  TREATMENT 

1 .  Wellness  -  How  sick  are  you  feeling  on  a  scale  of  1  - 10,  with  1  being  not  very  sick  and  10 
being  very  sick. 

123456789  10 

Not  Very  Very  Sick 

Sick 

2.  Pain  -  How  bad  is  your  pain  on  a  scale  of  1  -  10,  with  1  being  not  much  pain  and  10  being 
severe  pain. 


1  2  3  4  5 

Not  much 
Pain 


6  7  8 


9  10 

Severe 
Pain 
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PATIENT  WELLBEING  SURVEY 


Data  Specifications 


Data  Item  Descriptor 

Definition  and  Specification 

Notes 

Patient  Wellbeing  Survey 

•  Two  outcome-oriented  bipolar 
rating  questions.  Each  question 
(posed  verbally)  will  assess  for 
a  difference  pre  and  post 
treatment. 

•  The  respondent  may  be  asked 
either  question  #1,  question  #2, 
or  both  #1  and  #2.  The  survey 
asks  the  respondent  to  rate: 

(1)  how  sick  they  are 

(2)  how  severe  their  pain  is 

•  The  respondent  is  the  patient. 

•  The  triage  nurse  determines,  based  on 
the  patient’s  chief  complaint,  which 
questions  (#1,  #2,  or  both)  apply. 

With  this  tool  a  patient  may  be  sick 
but  not  in  pain  or  a  patient  may  be  in 
pain  from  an  isolated  injury  but  overall 
not  sick.  Additionally,  a  third 
scenario  exists  where  both  questions 
would  apply  to  a  patient  who  (for 
example)  presents  with  fever  and 
abdominal  pain. 

Data  Components: 

Judgement  is  indicated  on  a  10  point 
scale.  The  scale  value  is  circled  to 
record  the  respondent’s  judgment  of 
overall  sickness  or  pain. 

Data  Collection  Instrument 

Field-testing  underway 

Sampling  Scheme 

•  All  patients  in  which  the  triage 
nurse  (based  on  their 
assessment)  anticipates 
treatment  and  discharged  from 
the  urgent  area  of  care. 

•  The  respondents  (patients)  will 
be  surveyed  verbally  pre  and 
post  treatment: 

(1)  at  triage  and 

(2)  prior  to  discharge. 

•  Walk-In  pts.  excluded  from  RIH 
data  collection.  Walk-In  data 
has  not  been  used  in  any  of  RIH 
measurement  development  and 
sampling  schemes. 

The  questions  assess  for  relevant 
information.  The  ability  to  provide  a 
response  will  carry  the  potential  to 
ultimately  t  nurturing,  T  satisfaction,  and  t 
the  opportunity  for  the  team  to  support  pt. 
with  information  relating  to  their  state  of 
wellbeing. 
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Behaviorally  Anchored  Rating  Scales 


1.  Measure  Category: 

2.  Type  of  Measure: 

3.  Status  of  Development: 

4.  Data  Elements: 

5.  Data  Fields: 

6.  Comment: 


Team  Behavior 

Observational 

Under  development 

Fixed 

Form 

Rating  categories  have  been  determined.  Developmental  efforts 
surround  the  (a)  length  of  time  that  the  observer  needs  to  observe  a 
team,  (b)  which  teams  to  observe,  and  (c)  the  categories  of  patient 
acuity  or  chief  complaint  to  select  for  observing  team  behavior. 

The  narrative  description  of  the  rating  scales  appears  in  Appendix 
A. 
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Team  Dimensions  Rating  Form 

Use  the  instructions  in  the  MedTeams  Resource  Guide  to  Complete  this 

form 


Team: 


Shift: 


Date: 


No  Team  Dimension 


1  Team  Structure  and 
Climate 


2  Apply  Problem  Solving 
Strategies 


Support  Team  with 
Information 


Execute  Plans  and 
Manage  Workload 


5  I  Improve  Team  Skills 


Evaluator: 


!  Descriptors 


Establish  leader,  form  team,  delegate  team  role,  establish 
expectations,  demonstrate  respect,  issue  challenge,  explore 
alternatives,  use  constructive  exchanges,  accent  resolution 


Engage  members  in  planning,  identify  protocol  or  develop  plan, 
engage  team  in  decision  making,  alert  team  to  potential  biases 
and  errors,  report  slips,  lapses,  and  mistakes  to  team,  use 
advocacy  and  assertion,  apply  2  challenge  rule 


Employ  common  terminology,  call  out  requests  for  info,  use  check 
backs  to  verify  info,  systematically  handof  responsibilities  during 
transitions,  offer  info  to  support  decisions,  communicate 
decisions  to  team  members,  offer  information  to  support  planning, 
communicate  plans  to  team  members,  seek  information  for 
decision  making,  seek  info  for  planning 


Execute  protocol  or  plan,  resolve  deviations  from  protocol  or  plan, 
integrate  individual  assessments  of  patient  needs,  replan  patient 
care  in  response  to  overall  caseload,  prioritize  patient  tasks, 
prioritize  all  tasks  for  all  team's  patients,  balance  workload  within 
team,  request  help  for  task  overload,  offer  help  for  task  overload, 
use  periods  of  low  workload,  cross  monitor  actions,  redirect  team 
focus  back  to  team  and  clinical  tasks,  request  SA  updates, 
provide  SA  updates,  monitor  execution  of  protocol  or  plan 


Provide  situation-specific  teaching,  explain  previous  actions 
unexplained  due  to  urgency,  engage  in  team  review 


Notes:  Consult  the  behaviorally  anchored  rating  scales  for  details.  Enter  a  summary  rating 
(1 , 2  ...  7)  in  the  rating  block  for  each  Team  Dimension.  Refer  to  the  rating  scale  below. 


Very  Poor 
1 


Marginal 

3 


Rating  Scale 
Acceptable 

4 


Very  Good 
6 


Superior 

7 


Observed  Error  Record 


1.  Measure  Category: 

2.  Type  of  Measure: 

3.  Status  of  Development: 

4.  Data  Elements: 

5.  Data  Fields: 

6.  Comment: 


Team  Behavior 

Observational 

Under  development 

Fixed 

Form 

This  instrument  will  be  available  to  record  any  team  errors  that  data 
collectors  may  observe  while  collecting  other  data,  such  as  BARS 
assessments,  in  the  ED. 
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Observed  Error  Record  for  BARS  Assessment 


1.  Did  you  observe  any  errors?  _ Yes  _ No 

2.  If  you  answered  Yes  to  the  Question  1,  provide  a  brief  one  or  two  sentence  description  of 
what  you  observed. 

3.  Did  you  intervene?  _ Yes  _ No 

4.  If  you  answered  Yes  to  Question  #3,  what  actions  did  you  take? 

5.  In  your  opinion,  would  teamwork  have  eliminated  the  error  or  reduced  it? 

_ Yes  _ No 

6.  If  you  answered  Yes  to  Question  4,  which  BARS  behaviors  would  have  been  employed  to 
eliminate  or  reduce  the  error? 
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Subjective  Workload  and  Stress  Scales 


1.  Measure  Category:  Team  Behavior 

2.  Type  of  Measure:  Survey 

3.  Status  of  Development:  Completed 

4.  Data  Elements:  Fixed 

5.  Data  Fields:  Form  A.  Six  scale  NASA  Task  Load  Index 

Form  B.  Overall  Workload  and  Stress  Scale 


6.  Comment: 
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Subjective  Workload  Assessment 

(Form  A) 


Time 


Location 


Position 


Rate  your  workload  in  providing  care  to  this  patient  by  completing  the  following  six  scales. 


Place  an  X  on  the  vertical  line  of  each  scale 


i£i 


Mental  Demand  .  , 

How  mentally  demanding  was  the  care  you  provided?  Ill  1  I  11  1  I  II  II  1  11  11  11 

Very  Low  Very  High 


Physical  Demand  I  I  I  I  I  I  I  I  I  I 

How  physically  demanding  was  the  care  you  provided?  —I _ L_  1  1  I  I  1  1  _j _ [_ 

Very  Low  Very  High 


Time  Demand  I  I  I  I  I  I  I  11  II  I  I  I  I  _ _ 

How  hurried  or  rushed  was  the  care  you  provided?  , , 

Very  Low  Very  High 


Performance  I  1  I  II  I  I  I  I  I  I  I  I  I  I  I 

How  successful  were  you  in  accomplishing  the 

care  you  provided?  Failure  Perfect 


Effort 

How  hard  did  you  have  to  work  to  provide 
the  patient’s  care? 


Very  Low  Very  High 


Frustration 

How  insecure,  discouraged,  irritated  and  annoyed 

were  you  in  providing  the  patient’s  care? 


Very  Low  Very  High 
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Subjective  Workload  and  Stress  Assessment 
(Form  B) 


For  this  point  in  your  shift,  rate  your  workload  and  level  of  stress  using  the  scales  below. 


Overall  Workload 


Overall  Workload:  Rate  your  overall 
workload  for  all  the  patients  and  the 
responsibilities  you  have  at  this  moment. 
Place  an  X  on  one  of  the  vertical  lines  of 
the  scale.  1  [  I  i  1  i  I 


Very  Low  Very  High 


Stress  Scale 

Circle  one  word  that  best  describes  how  you  feel  right  now 

Wonderful 

Fine 

Comfortable 
Not  bothered 
Steady 
Unsteady 
Worried 
Uncomfortable 
Overwhelmed 
Melt  down 
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Form  B.  Overall  Workload  (OW) 

Data  Specifications 


Definition  and  Specifications 

Notes 

FormB.  Overall 
Workload  (OW) 

A  Bipolar  Rating  question:  asks  the 
respondent  to  judge  their  workload  along  an 
ordered  dimension.  Bipolar  ends  specify 
opposite  extremes  of  the  workload 
continuum.  The  number  of  gradations 
correlate  with  odd  numbers  to  allow  for 
neutral  point. 

Sampling  Scheme 

•  Collect  from  each  RN,  Dr.,  and  technician 
on  duty. 

•  Triage,  Trauma,  WI,  and  OU  gathered 
and  labeled  to  indicate  gathering  site. 

•  Attempts  made  to 
account  for  all  data 
points.  Respondents 
absent-on-duty  for  breaks 
and  meals. 

•  Distinguishing  individual 
gathering  sites  provides 

PI  with  the  ability  to 
determine  impact  of  site 
specific  data. 

Collection 

•  Administered  with  Stress  Scale 

•  OW  &  SS  time  to  administer  and  scored. 

•  Resources  Needed  -  administered  by 
research  assistant. 

•  Measure 

-produced  by  research  team, 
-scored  by  research  assistant, 
-date  entered  by  research 
assistant. 

-analyzed  by  PI. 

•  Time  Estimation: 

-10  seconds  per  respondent. 

-20  respondents/ 15  minutes. 
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FORM  B:  STRESS  SCALE  (SS) 
Data  Specifications 


Definition  and  Specifications 

Notes 

FormB:  Stress  Scale 

•  Measure  is  cafeteria  style  question: 
asks  respondents  to  “select  a  response 
that  most  closely  corresponds  to  their 
perspective”. 

•  Composite  Scale:  assigned  numerical 
values,  placed  on  a  continuum  with 
respect  to  attributes  being  measured. 

•  Quantitatively  discriminates  between 
respondents  with  differing 
perspectives. 

Sampling  Scheme 

•  Collect  from  each  RN,  Dr.,  and 
technician  on  duty. 

•  Triage,  Trauma,  WI  and  OU  gathered 
and  labeled  to  indicate  gathering  site. 

•  Attempts  made  to 
account  for  all  data 
points.  Respondent 
“absent  -  on-duty”  for 
breaks  and  meals. 

•  Distinguishing 
individual  gathering 
sites  provides  PI  with 
the  ability  to  determine 
impact  of  site  specific 
data. 

Collection 

•  Resources  Needed  -  administered  by 
research  assistant. 

•  Measure 

-produced  by  research  team, 
-scored  by  research  assistant. 

-data  entered  by  research 
assistant. 

-analyzed  by  PI. 

•  Time  Estimation  -  1 0  seconds  per 
respondent 
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Teamwork  Nursing  Assessment 


1.  Measure  Category:  Team  Behavior 

2.  Type  of  Measure:  Survey 

3.  Status  of  Development:  Completed 

4.  Data  Elements:  Fixed 

5.  Data  Fields:  Form 

6.  Comment: 
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TEAMWORK  NURSING  ASSESSMENT 

(TNA) 


Record  the  time  delay  between  the  point  you  believed  the  patient  would  be  admitted 
and  when  the  ED  physician  communicated  the  admit  decision  to  you. 

Date _  Communication  Delay: _ hrs. _ min. 

(If  no  delay,  enter  0) 

Did  the  delay  in  communication  cause  a  delay  in  the  patient’s  admission?  I  [Y  I  |N 
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TEAMWORK  NURSING  ASSESSMENT 


Data  Specifications 


Data  Item  Descriptor 

Definition  and  Specifications 

Note 

Teamwork  Nursing 
Assessment  (TNA) 

Objective  Communication  Measure: 

Asks  two  questions: 

1 .  Is  there  a  communication  issue;  and  if 
so 

2.  Does  that  issue  impact  operations. 

Assesses  for  TD3,  Communicate 
with  Team,  of  the  ETCC™Team 
Dimensions 

Sampling  Scheme 

Collect  for  all  patients  admitted  where  a 
difference  exists  between  time  points  of 
RN  to  Dr.  communication. 

Monitoring  admissions  and 
prompting  of  professionals  is 
essential  initially  to  assure 
reliable  outcomes. 

Collection 

•  Resources:  respondent  self 
administers. 

•  Measure: 

-produced  by  research  team, 
-scored  by  research  assistant, 
-analyzed  by  PI. 

•  Time  Estimation:  Dependent  on 
number  of  patient  admissions.  10 
minutes  per  each  10  nurses 
(example  80  minutes  per  10  nurses 
for  an  eight  hour  shift). 

•  Hypothesis:  As  self 

prompting  to  record  each 
admission  event  increases  the 
aggregate  data  collection 
time  will  decrease. 
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Pedometer  Studies 


1.  Measure  Category:  Team  Behavior 

2.  Type  of  Measure:  Observational 

3.  Status  of  Development:  Under  development 

4.  Data  Elements:  Fixed 

5.  Data  Fields:  Form 

6.  Comment: 
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ETCC™  Course  Evaluation 


1.  Measure  Category:  Team  Behavior 

2.  Type  of  Measure:  Program  Evaluation 

3.  Status  of  Development:  Completed 

4.  Data  Elements:  Fixed 

5.  Data  Fields:  Form 

6.  Comment: 
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Emergency  Team  Coordination  Course 

Course  Evaluation 

Name:  _ _ (optional)  Date: _ 

A.  Listed  below  are  the  program  objectives.  Please  rate  the  extent  to  which  you 
are  now  able  to: 

Low  High 

i  t  i  i*r  .1  ■%  r+ .  «  *  .  M  ° 


1  Identify  the  benefits  and  key  elements  of  teams.  1  2  3  4  5 

2  Describe  team  structure  and  climate  as  it  relates  to  the  emergency  environment  1  2  3  4  5 

3  Describe  several  problem-solving  strategies  that  can  be  applied  in  the  emergency  1  2  3  4  5 

environment. 

4.  Discuss  ways  to  execute  plans  and  manage  workload  in  the  emergency  environment.  1  2  3  4  5 

5.  Describe  how  to  support  the  emergency  department  team(s)  with  information  and  actions.  1  2  3  4  5 

6.  Identify  methods  to  improve  team  skills  1  2  3  4  5 

7.  Discuss  the  clinical  integration  of  team  coordination  in  the  emergency  environment.  1  2  3  4  5 

8.  Employ  a  variety  of  team  coordination  skills  in  the  emergency  environment.  1  2  3  4  5 


B.  Content: 


Related  to  Objectives 

12  3  4  5 

Well  Organized 

1  2  3  4  5 

C.  Teaching  Strategies: 

Effective  Teaching  Methods 

1  2  3  4  5 

For  the  following  ratings ,  1  =  poor  and  5  -  excellent 


D.  Faculty 

Instructor  A 

Instructor  B 

Instructor  C 

Presentation  Style  1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

Knowledge  of  Subject  1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

E.  Physical  Facilities  1  2  3  4  5 

F.  How  will  you  use  the  information  you  received  today  to  improve  your  practice? 
H.  Comments: 
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Emergency  Team  Coordination  Course 
Course  Evaluation  (continued) 


I.  Other  Ratings 

1 .  Time  available  for  the  instructor  to  adequately  present  and  discuss  each 
team  dimension  topic. 


Not  enough 
time 

1  2 


Too  much 
time 

3  4  5 


2.  Pace  of  instruction 


Too  slow  Too  fast 

1  2  3  4  5 


3.  Vignettes  presented  realistic  problems 


Unrealistic 

Realistic 

1  2 

3  4  5 

4.  Vignettes  helped  me  to  understand  how  to  apply  the  team  dimensions. 


Not  Helpful  Helpful 

1  2  3  4  5 


5.  Team  coordination  techniques  I  learned  today  will  become  an  integral  part  Disagree 
of  my  daily  ED  activities  t 


Agree 

3  4  5 


6.  After  attending  this  training,  rate  the  usefulness  of  this  course. 


Of  slight  use  Very  useful 

1  2  3  4  5 


7.  Rate  the  format  of  the  student  workbook 


Poor  Excellent 

1  2  3  4  5 


54 


Resource  Requirements  for  Data  Collection 


This  section  contains  information  on  the  resources  required  for  data  collection.  As 
instruments  were  administered  at  RIH,  a  journal  was  kept  of  the  time  required  to  present  an 
instrument  to  respondents,  the  time  required  for  the  respondent  to  complete  the  instrument,  and 
the  time  required  to  score  the  instrument  or  record  its  data  elements.  These  basic  data  form  the 
basis  for  making  estimates  for  the  amount  of  time  necessary  to  collect  and  record  data  during  the 
actual  implementation  of  the  measurement  suite. 

Table  3  provides  a  listing  of  four  factors  associated  with  data  collection.  The  factor  What 
refers  to  the  target  of  data  collection  such  as  the  individual  patient  or  the  ED  as  a  whole.  The 
factor  Who  lists  the  person  or  functional  entity  most  likely  to  collect  the  data.  The  third  factor, 
When,  is  how  frequently  the  data  are  collected.  The  final  factor.  How,  lists  the  various  forms  in 
which  the  data  are  reported  or  available  for  use. 

Two  important  findings  emerge  from  the  data  in  Table  3.  The  first  is  that  a  considerable 
amount  of  data  of  interest  to  the  validation  effort  is  probably  collected  by  automated  systems. 
These  data  can  be  entered  into  the  data  base  through  electronic  means.  The  second  is  that  most 
of  the  instruments  developed  for  the  validation  can  be  administered  by  a  research  assistant  trained 
in  their  administration  and  scoring.  The  expertise  of  a  physician  or  nurse  is  required  only  for  (1) 
observing  and  recording  team  behavior  using  the  Behaviorally  Anchored  Rating  Scales  and  (2)  for 
recording  clinical  or  procedural  errors  using  the  Observed  Error  Record. 

In  order  to  develop  an  estimate  of  the  manhours  required  to  collect  a  month’s  worth  of 
data  at  a  typical  validation  site,  a  scenario  was  created  that  integrates  information  gathered  on  the 
length  of  time  needed  to  collect  data  with  information  provided  in  Table  3.  The  assumptions  of 
the  data  collection  scenario  are  the  following 

•  A  research  site  will  employ  a  research  assistant  for  data  collection 

•  The  site  does  not  have  automated  support  for  length  of  stay  data.  The  research 
assistant  will  review  50  medical  records  a  day  for  30  days  to  obtain  length  of  stay 
data  and  other  timed  data. 

•  A  physician  or  nurse  will  observe  teams  and  provide  behaviorally  anchored  rating 
scale  data 

•  The  research  assistant  has  personal  computer  support  for  transferring  data  to  a  data 
base. 


Tables  4  and  5  show  the  manhour  estimates  for  collecting,  recording,  and  entering  data  into  a  data 
base  assuming  a  frequency  of  survey  instrument  administration  at  the  moderate  level.  Table  5 
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Table  3.  Factors  Associated  with  Collecting  Measurement  Suite  Data. 


Collection  Factors 


Actions 


Measure  or  Instrument 


Performance _ 

Overall  LOS _ 

Throughput  Time  Measures 
Subclassifications  of  LOS 
(admit/disch,  teams,  traumas, 
billing  codes,  ED  sections) 


Timed  Procedures  or  Events 

TPA-Stroke _ 

Chest  Pain  Evaluation _ 

Time  to  T reatment  (charcoal, 
antibiotics  for  menigitis  or  septic 
children,  asthma) 


What 

Who 

When 

How 

IP 

A,  RA 

D 

A,  MRR 

IP 

A,  RA 

D 

A,  MRR 

IP 

A,  RA 

D 

A,  MRR 

IP 

RA 

IE 

MRR 

IP 

RA 

IE 

MRR 

IP 

RA 

IE 

MRR 

Time  to  see  MD  perhaps  best 
measure 


Quality  Indicators 
AMA 

LWBS  | 

Left  Before  T reatment  Completed 


Mislabeled  labs 


Mislabeled  Bloods 


Blood  culture  contamination  rate 
Medication  errors  I  ~ 

Incident  Reports _ 

Patient  Complaints _ 

Pt  Compliments 


Returns  within  48  or  72  hrs 


T reatment  Delays  -  Consults 


IRA  transfers  data  to  database 


Administrative 

Sick  Leave 

ED 

HAO 

M 

AR 

Tardiness 

ED 

HAO 

M 

AR 

Unexcused  Absences 

ED 

HAO 

M 

AR 

Staff  Turnover 

ED 

HAO 

M 

AR 

Nurses,  techs,  physicians 

Trauma  Registry  (Subset  used  for  initial  analyses)  | 

Arrival  date  at  ED 

IP 

Arrival  time  in  ED 

IP 

Discharge  time  from  ED 

IP 

Glasgow  scale  on  ED  arrival 

ip 

Revised  T rauma  Score  on  ED  arriv 

IP 

ED  disposition 

ip 

Days  in  the  ICU 

IP 

Days  in  the  hospital 

IP 

GCS  <  13,  no  CT 

IP 

Cerv  spine  fracture  missed 

IP 

RA  W 

AR 
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Table  3.  Factors  Associated  with  Collecting  Measurement  Suite  Data  (continued). 


\Attitudes  and  Opinion 

Maslach  Burnout  Surve 

y 

IS 

RA 

SE 

DRCF 

Twice  bef( 

)re,  then  post? 

Staff  Attitudes 

IS 

RA 

SE 

DRCF 

Patient  Satisfaction 

IP 

RA 

SE 

LF,  TS 

Review  and  adapt  lo< 
instruments  for  team\ 
or  wellness  questions 

:al 

work  items 

Admission  Evaluation  S 
Patient  Wellbeing  Surv< 

iurvey 

IP 

RA 

AAP 

DRCF 

Review  and  revise  for  local  use 

sy 

IP 

RA 

AP 

DRCF 

Survey  most  likely  to  be  discha'd 

Team  Behavior 

Behaviorally  Anchored  Rating  Seal 

IT 

“SME 

SE 

DRCF 

Teamwork  Nursing  Assessment 

IP 

RA 

SE 

DRCF 

£>uDjective  Workload 
(full  TLX  and  OWL) 

IS 

RA 

SE 

DRCF 

Stress  Scale 

IS 

RA 

SE 

H33ai 

Pedometer  studies 

IS 

Ira 

mm 

[iktwtai 

MiMffi 

Program  Evaluation 

ETCC  Course  Critique 

IS 

RA 

SE 

DRCF 

Dept  Demographics 

Number  of  visits 

ED 

RA 

D 

AR 

Number  of  admissions 
to  inpatient  units 

ED 

RA 

D 

AR 

Census  per  day 

ED 

RA 

D 

AR 

Net  revenue  per  visit 

ED 

RA 

M 

AR 

Total  expense  per  visit 

ED 

RA 

M 

AR 

Total  cost  per  visit 

r* - 

ED 

RA 

M 

AR 

Material  co: 

_ L 

st  only 

bbreviation  Codes 


What:  ED  =  emergency  department  IP  =  individual  patient  IS  =  individual  staff  member  IT  = 
individual  team 

Who:  A  =  automated  system  HA  =  hosptial  administration  QAO  ■  quality  assurance  office 
SME  =  subject  matter  expert  RA  *  research  assistant 

When:  AP  =  all  patients  AAP  =  all  admitted  patients  D  =  daily  IE  =  individual  event  M  = 
monthly  SE  =  scheduled  event  (specific  schedule)  W  =  weekly 

How:  A  =  automation  AR  =  administrative  report  DRCF  =  Dynamics  Research  Corp  form  LF 

*  local  form  MRR  =  medical  record  review  QAR  *  quality  assurance  report  TS  *  telephone 
survey 
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Table  4.  Scenario  Example:  Time  Estimate  for  Research  Assistant  to  Gather 

Data 


Measurement 

Activity 

Subareas 

Time  Estimate 
(Hours) 

Scheduled  time  in 
ED  to  collect  data 

3  hours  per  day  for  30  days 

90.0 

ED  Performance 
Collection 

Quality  Assurance 

Admin/Personnel 

Other  Operations  Data 

Length  of  Stay  and  Other  Intermediate  Times 
(see  data  entry) 

Timed  Procedures  (see  data  entry) 

10.0 

Attitudes, 

Opinion,  and 

Team  Behavior 
Collection 

Admission  Eval  Survey  (250  x  .25  min) 
Maslach  (100  x  2  min) 

Subjective  Workload  (100  x  1  min) 

Staff  Attitudes  (100  x  2  min) 

TNA  (100  x  .10  min) 

Pedometer  Readings  (50  x  5  min) 

Patient  Wellbeing  (300  x  1  min) 

Patient  Satisfaction  (see  data  entry) 

1.0 

3.3 

1.7 

3.3 

0.2 

4.2 

5.0 

18.7 

GRAND  TOTAL 

118.7 
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Table  5.  Scenario  Example:  Time  Estimate  for  Research  Assistant  to  Score  and 

Enter  Data 


Measurement 

Area 

Subareas 

Time  Estimate 
(Hours) 

ED  Performance 

Quality  Assurance 

Admin/Personnel 

Other  Operations  Data 

Length  of  Stay  and  Other  Intermediate  Times 
(30  days  x  50  MR/day  x  2  min/MR) 

Timed  Procedures  (30  forms  x 

2min/form) 

4.0 

1.0 

4.0 

50.0 

1.0 

60.0 

Attitudes, 

Opinion,  and 

Team  Behavior 

Admission  Eval  Survey  (250  x  1  min) 

Maslach  (100  x  3  min) 

Subjective  Workload  (100  x .  10  min) 

Staff  Attitudes  (100  x  3  min) 

TNA  (100  x  .10  min) 

Pedometer  Readings  (50  x .  10  min) 

Patient  Wellbeing  (300  x  ,40min) 

Patient  Satisfaction  (250  x  1.5  min) 

4.2 

5.0 

1.3 

5.0 

0.2 

0.1 

2.0 

6.3 

24.1 

GRAND  TOTAL 

84.1 

Table  6.  Scenario  Example:  Total  Times  for  Data  Collection,  Scoring,  and  Entry 


Researcher 

Data  Activity 

Time  Estimate 

Physician  or  Nurse 

Behaviorally  Anchored  Rating  Scales. 

30  1.0  hour  observations 

20  0.5  hour  observations 

40  hrs 

Research  Assistant 

ED  Performance,  Attitudes  and  Opinion, 
Other  Team  Behavior 

203  hrs 

TOTAL 

243  hrs 
(30  days) 

59 


summarizes  the  manhours  required  for  the  physician  or  nurse  and  research  assistant  data 
collection  team.  This  scenario  indicates  that  a  full-time  research  assistant  will  be  required  at  the 
typical  site  to  collect  data  for  the  full  validation  suite  of  instruments.  The  physician  or  nurse  will 
be  required  to  complete  40  hours  of  data  collection  during  the  same  period.  Note  that  automated 
support  for  collecting  length  of  stay  data  will  reduce  the  data  collection  effort  by  50  hours.  Other 
automated  data  sources  could  reduce  this  “worse  case”  scenario  manhour  estimate  further. 
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APPENDIX 

Behaviorally  Anchored  Rating  Scales 


A-l 


Emergency  Medical  Team  Dimension  1:  Maintain  team  structure  and  climate 
Overview 

This  rating  assesses  the  quality  of  professional  relationships  among  physicians,  nurses, 
and  other  personnel  comprising  the  emergency  care  team,  and  the  overall  interpersonal 
climate  of  the  emergency  department  Team  members  acknowledge  the  essential  team 
objective—provide  the  highest  standard  of  patient-centered  care.  The  team's  goals  are 
both  patient-centered  and  task-centered  with  the  focus  on  achieving  both  goals  rather 
than  who  specifically  accomplishes  them. 

The  team  leader  in  emergency  medicine  is  the  attending  physician  who  has  final 
authority  in  clinical  matters.  The  senior  attending  physician  sets  the  tone  of  the  team 
and  maintains  a  cooperative  working  environment  by  sharing  or  delegating  authority 
and  promoting  the  participation  of  all  team  members. 

This  rating  also  concerns  the  importance  of  the  timely  dissemination  of  information 
essential  to  team  formation  and  coordination.  Team  formation  consists  of  learning  who 
is  serving  on  the  team  and  knowing  each  team  member's  responsibilities,  becoming 
aware  of  the  clinical  status  of  the  team's  patients,  and  knowing  the  operational  issues  in 
the  emergency  department  and  elsewhere  affecting  overall  operations.  All  team 
members  communicate  essential  information  through  face  to  face  contact  or  other 
means. 

Because  emergency  care  is  a  team  effort,  each  member  recognizes  his  or  her  own 
special  skills,  and  acknowledges  and  appreciates  the  skills  of  other  team  members. 
Team  members  understand  their  interdependence  and  demonstrate  a  willingness  to 
assume  responsibility,  act  autonomously,  and  offer  assistance  to  other  team  members. 
Team  members  assist  each  another  in  achieving  the  highest  technical  competence  and 
advocating  respect  for  patients  and  coworkers.  While  the  physician  has  clear  lines  of 
clinical  authority  and  responsibility,  team  leadership  is  flexible  since  situations  may 
require  any  team  member  to  assume  situational  leadership.  Decisions  are  supported  by 
the  team  even  when  there  may  not  be  complete  agreement 


Superior  Rating  (7) 

Each  team  member  assumes  responsibility  for  becoming  aware  of  relevant  clinical  and 
operational  issues.  The  attending  physician  and  a  nursing  team  member  ensure  that 
actions,  duties,  and  task  responsibilities  are  partitioned  and  clearly  assigned  to  specific 
individuals.  Questions  and  discussion  about  tasks  and  specific  responsibilities  are 
encouraged. 
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The  team  members  have  very  good  interpersonal  relationships.  They  respect  others 
skills  and  appear  to  enjoy  being  with  each  other.  There  is  a  genuine  concern  for  good 
working  relationships.  Team  members  go  beyond  common  workplace  courtesies  to 
express  appreciation  of  other  member's  contributions  and  to  reinforce  good  teamwork. 
No  degrading  comments  or  negative  voice  tones  are  used  in  interactions.  The  climate  is 
very  open;  team  members  freely  talk  and  ask  questions.  Team  members  encourage  the 
individual  with  the  most  information  about  the  situation-at-hand  to  participate. 
Disagreements  are  perceived  as  a  normal  part  of  team  interactions,  and  the  team 
directly  confronts  the  issues  over  which  the  disagreement  began.  Arguments  or 
disagreements  focus  on  the  specific  situation  behavior  or  solution  rather  than  on 
personalities.  Each  team  member  carefully  listens  to  others'  comments.  Senior  team 
members  accept  challenges  from  junior  team  members.  Alternative  solutions  are 
explored.  The  solution  produced  is  a  "win-win"  situation  in  which  all  team  members' 
opinions  are  considered.  The  team  members  have  no  hard  feelings  at  the  conclusion  of 
the  incident  Offers  of  assistance  are  freely  given  and  appreciatively  acknowledged. 
Each  team  member  demonstrates  patient-centered  care  attitudes.  Team  members 
monitor  the  attitudes  of  other  team  members  and  offer  positive  and  negative  feedback. 


Acceptable  Rating  (4) 

A  brief  description  of  the  current  emergency  department  status  is  provided  to  some  or 
all  members  of  the  team.  The  attending  physician  and  nursing  team  members  ensure 
that  task  responsibilities  are  partitioned  and  assigned  to  specific  individuals.  Routine 
assignments  are  adequately  covered,  but  contingencies  for  special  situations  are  not 
carefully  planned.  One  or  more  team  members  seek  necessary  information  during  the 
course  of  organizing  the  team,  but  some  team  members  remain  less  well  informed  than 
others. 

The  team  members  have  sound  interpersonal  relationships  and  seem  to  respect  one 
another's  skills.  However,  staff  members  interact  on  clinical  and  operational  issues  in  a 
business-like  manner  and  engage  in  only  limited  socially-oriented  interchanges. 
Although  team  members  are  courteous,  the  tone  of  interactions  is  formal  and 
businesslike.  The  climate  is  an  open  one,  and  team  members  are  free  to  talk  and  ask 
task-oriented  questions.  Regardless  of  rank,  duty  position,  or  professional  status, 
individuals  with  information  about  the  situation-at-hand  are  allowed  to  participate. 
When  disagreements  arise,  the  team  directly  confronts  the  issues  over  which  the 
disagreements  began.  The  primary  focus  is  on  behavior  or  solutions,  and  no  personal 
attacks  are  made  in  the  heat  of  discussion.  The  solution  is  generally  seen  as  reasonable. 
Problem  resolution  ends  on  a  positive  note  with  very  little  hostility  or  grumbling 
among  team  members.  Mutual  respect  is  clearly  intact  Team  members  generally 
remain  focused  on  their  assigned  tasks,  but  assistance  is  offered  as  conditions  allow. 
Obvious  disregard  of  patient-centered  attitudes  is  not  apparent  Team  members  correct 
instances  of  improper  patient-centered  attitudes. 
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Very  Poor  Rating  (1) 


A  team  member  provides  some  or  all  team  members  with  essential  work-related 
information  with  little  or  no  attendant  explanation.  There  is  little  or  no  discussion  of 
responsibilities  or  their  assignments  to  specific  team  members.  Team  members  tend  not 
to  ask  questions  about  the  operations  for  their  shift  If  asked,  questions  tend  to  be  cut 
off,  only  briefly  addressed,  or  ignored  by  the  other  team  members. 

Team  interactions  are  often  awkward  and  uncomfortable.  The  team  members  do  not 
appear  to  like  or  respect  each  other.  Team  members  may  be  curt  and  impolite  to  each 
other.  Requests  for  assistance  are  made  as  commands  rather  than  as  requests  for 
support  When  disagreements  arise,  the  team  fails  to  directly  confront  the  issues. 
Personal  attacks  may  arise.  Senior  team  members  are  resistant  to  recommendations 
from  junior  team  members.  Team  members  do  not  explore  the  range  of  possible 
solutions.  They  may  shout  or  argue  without  finding  a  solution.  One  or  more  team 
members  may  retreat  and  say  nothing  at  all.  A  "win-lose"  situation  develops  in  which 
one  team  member  is  shown  to  be  right  and  the  other  to  be  wrong.  The  team  members 
show  little  respect  to  one  another  except  for  deferring  to  formal  rank  or  professional 
status.  Assistance  may  be  withheld  or  provided  only  in  cases  of  extreme  need.  Team 
members  exhibit  attitudes  that  may  jeopardize  patient  welfare. 


Emergency  Medical  Team  Dimension  2:  Apply  Problem-solving  Strategies 

Overview 

This  rating  evaluates  the  team's  ability  to  apply  established  algorithms,  protocols,  and 
other  preplanned  actions,  and  to  demonstrate  flexibility  in  modifying  these  or  planning 
for  emergent  situations.  The  quality  of  the  team's  engagement  in  the  decision-making 
and  planning  processes  is  assessed.  The  openness  to  contributions  of  information  or 
ideas  from  team  members  is  evaluated. 

The  team  engages  in  planning  activities  required  for  unusual  situations  in  patient  care, 
and  carries  out  established  routines  and  practices  for  anticipated  clinical  and 
administrative  situations  in  the  emergency  department  Planning  for  unusual  situations 
involves  collecting  essential  information,  identifying  potential  problems  and  courses  of 
action,  assessing  risks,  and  determining  required  tasks  and  assigning  specific 
responsibilities  to  each  team  member.  Planning  also  may  include  mental  rehearsal  or 
practice  of  the  anticipated  course  of  action.  Team  members  also  implement  department¬ 
wide  plans,  protocols,  and  algorithms  covering  routine  operational  and  clinical 
situations. 

Although  the  physician  or  situational  leader  is  responsible  for  leading  planning 
activities,  this  rating  evaluates  the  extent  and  manner  in  which  the  entire  team 
participates.  Also,  the  time  constraints  on  the  team  is  considered.  If  there  was 
insufficient  time  to  conduct  comprehensive  planning  and  rehearsal,  the  team  is 
assessed  on  its  planning  and  rehearsal  of  the  most  critical  segments  of  a  significant 
situation.  That  is,  either  before  the  situation  or  while  it  was  unfolding,  did  the  team 
address  the  most  important  issues  given  the  time  available?  Note:  The  relationship 
among  team  members  should  be  observed  during  this  period  but  the  team  climate 
evaluation  should  be  made  on  rating  Emergency  Medical  Team  Dimension  1,  Maintain 
Team  Structure  and  Climate. 

Factors  to  consider  in  making  an  evaluation  of  decision-making  include  (a)  information 
available  to  the  team  members,  (b)  time  urgency  of  the  decision,  and  (c)  level  of 
involvement  and  information  exchange  among  the  team  members.  The  time-critical 
demands  of  emergency  medicine  require  many  decisions  to  be  made  on  an  event- 
driven  basis  with  only  a  minimum  level  of  information  exchange.  However,  when 
adequate  time  and  information  are  available,  team  members  are  expected  to  engage  in 
a  more  deliberate  and  interactive  style  of  decision  making.  The  evaluation  of  team 
decision  making  performance  should  ask  the  following  questions: 

1.  Did  the  team  use  all  of  the  available  information? 

2.  Was  the  type  of  decision  process  (  event-driven  versus  analytical) 
appropriate  for  the  time  available? 
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3.  Was  the  level  of  information  exchange  among  team  members  appropriate 
and  sufficient  for  the  decision  process  used? 

The  actions  of  team  members  in  establishing  and  maintaining  an  error-free 
environment  is  included  in  this  assessment  Team  members  understand  that 
judgemental  biases  impede  effective  decision-making,  and  that  slips,  lapses  and 
mistakes  are  naturally-occurring  events  instrumental  to  substandard  care.  Failures  of 
team  members  to  correct  specific  instances  of  biased  decision  making  or  errors  are 
assessed  in  this  team  dimension.  That  is,  the  absence  of  error-correcting  behavior  when 
one  or  more  team  members  is  aware,  or  should  be  aware,  of  erroneous  behavior  is 
assessed  in  this  team  dimension.  The  team's  proactive  use  of  emergency  department 
error-avoidance  systems  and  procedural  safeguards  is  also  assessed.  However,  the  use 
of  situation-specific  team  actions  to  intervene  to  correct  an  error,  such  as  cross¬ 
monitoring  or  maintaining  situational  awareness,  is  assessed  under  the  appropriate 
team  dimensions. 

Finally,  this  rating  evaluates  the  extent  to  which  team  members  advocate  a  course  of 
action  they  consider  best,  even  when  it  may  differ  with  the  one  being  followed  or 
proposed.  Note:  Except  under  extreme  conditions  where  time  is  absolutely  critical,  it  is 
usually  in  the  team's  best  interest  to  hear  the  full  range  of  viewpoints  available. 
Invoking  the  Two-Challenge  Rule  is  an  action  assessed  in  this  team  dimension. 


Superior  Rating  (7) 

The  team  acquires  new  and  updated  information  and  uses  it  to  develop  or  modify  the 
plan  of  action.  All  actions,  duties,  and  task  responsibilities  of  the  plan  are  partitioned 
and  clearly  assigned  to  specific  individuals.  Questions  and  discussion  about  the 
situation  and  specific  responsibilities  are  encouraged.  Potential  problems  are  noted  and 
discussed.  Courses  of  action  and  individual  responsibilities  are  established  in  the  event 
that  potential  problems  actually  occur.  The  physician  or  situational  leader  leads  the 
team  in  mentally  rehearsing  or  practicing  the  plan  by  visualizing  and  talking  the  team 
through  potential  problems  and  contingencies.  Team  members  acknowledge 
understanding  their  assigned  responsibilities  and  cues  for  action. 

Team  decision-  making  consistently  reflects  proper  attention  to  available  information. 
The  level  of  team  participation  and  deliberate  analysis  of  options  is  appropriate  for  the 
decision  time  available.  Resulting  decisions  are  timely  and  appropriate  given  the  time 
urgency  and  level  of  information  available  in  each  situation.  Team  members 
effectively  use  existing  error-avoidance  systems  and  procedures,  and  anticipate  or 
actively  corrrect  emergent  error  situations.  Team  members  appear  motivated  to  seek 
the  most  effective,  unbiased,  and  safe  decision  in  each  situation.  The  team  decides  and 
implements  a  course  of  action  before  the  situation  jeopardizes  team  performance  or 
patient  welfare. 
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Team  members  state  to  the  rest  of  the  team  a  course  of  action  that  they  consider  best. 
They  clearly  explain  their  reasons  for  believing  this  to  be  the  best  course.  Other  team 
members  listen  to  the  argument  before  presenting  any  criticism  or  proposing  alternate 
courses.  Discussions  focus  on  the  strengths  and  weaknesses  of  the  proposed  course  of 
action,  not  on  the  professional  status  or  personality  of  the  team  member  who  proposed 
the  action.  Other  team  members  expect  such  open  comments  and  view  them  as  positive 
contributions  to  team  performance.  The  two-challenge  rule  is  an  accepted  method  for 
avoiding  serious  actions  by  any  team  member. 


Acceptable  Rating  (4) 

A  brief  description  of  the  situation  is  provided  to  the  entire  team.  Responsibilities  are 
partitioned  and  assigned  to  specific  individuals.  Actions  are  taken  to  update  current 
information  that  adds  to  the  situation  briefing  and  helps  develop  the  plan.  One  or  more 
team  members  make  comments  during  the  course  of  developing  the  plan.  Potential 
problems  are  only  briefly  discussed.  There  is  adequate  preparation  for  contingencies. 
Team  members  briefly  discuss  the  operational  risks  in  the  plan.  Mental  rehearsal  or 
practice  is  initiated  by  the  physician  or  another  team  member  who  talks  through 
potential  problems  or  contingencies  for  one  or  more  aspects  of  the  intended  action. 
Some  discussion  takes  place  to  clarify  responsibilities  in  the  event  of  unexpected 
problems  or  contingencies. 

Team  decisions  sometimes  reflect  a  reluctance  to  share  or  use  available  information. 
Occasionally,  team  members  dwell  too  long  on  some  issues  while  neglecting  more  time 
urgent  requirements.  Most  decisions  are  timely,  but  on  occasion  are  not  well  resolved 
or  understood  by  the  team.  Most  decisions  are  appropriate  for  the  situation  with  the 
team  occasionally  overlooking  one  or  more  factors  or  options.  Occasionally,  team 
members  do  not  recognize  or  exploit  opportunities  for  additional  planning  or  rehearsal, 
substituting  ad  hoc  strategies  or  plans.  Team  members  avoid  errors  by  using  systems, 
procedures,  and  team  actions.  Decision  biases  are  avoided.  The  situation  may  worsen 
slightly  without  seriously  degrading  patient  treatment  before  the  team  decides  and 
implements  a  course  of  action. 

Team  members  state  their  support  for  a  course  of  action  or  suggest  improvements  to 
other  proposed  actions.  Each  team  member  makes  an  effort  to  explain  his  or  her 
position  and  convince  others  to  concur  on  the  course  of  action  to  be  taken.  Other  team 
members  may  counter  with  their  views  and  alternatives.  Team  members  usually  speak 
out  when  they  recognize  a  risky  departure  from  standard  procedures  or  when  they 
have  a  piece  of  information  that  is  important  to  another's  task  execution.  Other  team 
members  view  such  comments  as  constructive  and  not  as  a  challenge  to  authority.  The 
two-challenge  rule  is  a  formal  policy  reserved  for  only  the  most  serious  cases  of 
impaired  capacity  for  team  members. 
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Very  Poor  Rating  (1) 


The  physician  briefs  the  situation  with  little  or  no  attendant  explanation.  There  is  little 
or  no  discussion  of  specific  team  member  responsibilities.  Team  members  tend  not  to 
ask  questions  about  the  situation.  If  asked,  questions  tend  to  be  cut  off,  only  briefly 
addressed,  or  ignored  by  the  other  team  members.  Little  or  no  mention  is  given  to 
potential  problems  or  complications.  No  team  member  says  anything  about  operational 
risks  or  weaknesses  in  the  plan.  Any  suggestion  to  talk  through  a  potential  problem  or 
rehearse  responsibilities  is  rejected  as  unnecessary.  Interactions  are  abrupt  and 
impersonal. 

Planning  reflects  an  inflexible  style  of  decision  making  (either  event-driven  or 
analytical)  despite  time  urgency.  Team  members  may  engage  in  excessive  deliberation, 
overlook  the  relative  urgency  of  competing  decision  requirements,  or  act  impulsively. 
As  a  result,  decisions  frequently  lack  timeliness,  ignore  important  factors,  or  appear  out 
of  context.  Information  exchange  and  team  member  interaction  is  minimal,  so  that 
critical  input  is  ignored  or  not  sought  Serious  errors  may  be  captured  before  they  have 
serious  consequences,  but  error-avoidance  for  less  serious  errors  is  not  adequate. 
Decision  biases  may  not  be  challenged  .  Theteam  may  be  unable  to  decide  or 
implement  a  course  of  action  before  a  situation  becomes  critical. 

Except  for  the  team  leader,  team  members  almost  never  suggest  a  course  of  action. 
Team  members  attempting  to  propose  a  course  of  action  may  be  cut  off  before  they  can 
propose  the  action  or  explain  the  rationale  for  that  action.  Team  members  proposing 
courses  of  action  may  receive  personal  attacks.  Team  members  may  even  fail  to 
intervene  when  risks  such  as  incorrect  procedures  or  using  faulty  equipment  arise.  The 
two-challenge  rule  is  not  involked  by  team  members. 


Emergency  Medicine  Team  Dimension  3:  Communicate  with  the  Team 


Overview 

This  rating  covers  the  assessment  of  the  quality  of  information  exchange  within  the 
team  and  the  degree  of  reciprocity  among  team  members  in  giving  and  receiving 
information  and  assistance.  The  rating  also  includes  an  evaluation  of  the  team's 
adequacy  in  obtaining  information  from  patients  and  providing  information  to 
patients. 

Completeness,  timeliness,  and  quality  of  information  transfer.  Consider  both  oral 
and  written  communications.  Rate  the  information  value  and  clarity  of  team  member 
communication  in  terms  of  its  timeliness,  completeness,  and  avoidance  of  ambiguity 
(e.g.,  the  use  of  standard  terminology).  Include  in  the  evaluation  the  use  of  call-outs, 
check  backs,  feedback,  and  acknowledgments  to  verify  information  transfer  and  hand- 
offs  to  ensure  coordination  of  patient  care.  Evaluate  the  quality  of  instructions  and 
statements  associated  with  a  patient' s  evaluation  and  emerging  treatment  plan.  Also, 
take  into  consideration  information  exchange  with  patients.  Patient-oriented  examples 
include  obtaining  essential  history  and  symptom  information,  updating  patients  on  the 
status  of  their  diagnosis  and  treatment,  and  providing  discharge  instructions. 

Supporting  information  and  actions  sought  from  the  team.  Rate  the  extent  to  which 
team  members,  particularly  the  physician  or  situational  leader,  seek  supporting 
information  and  actions  from  the  team.  Evaluate  the  degree  to  which  team  members 
raise  questions  regarding  plans,  revisions  to  plans,  actions  to  be  taken,  and  the  status  of 
key  information.  Note:  The  extent  to  which  team  members  contribute  to  decision 
making  should  be  observed  here  but  evaluated  under  Emergency  Medicine  Team 
Dimension  2:  Apply  Problem  Solving  Strategies. 

Supporting  information  and  actions  offered  by  the  team.  Rate  the  extent  to  which 
team  members  anticipate  and  offer  both  supporting  information  and  actions. 
Supporting  information  is  offered  by  team  members  to  the  decision  maker  or 
situational  leader  evaluating  a  situation  or  preparing  to  make  a  decision.  Team 
members'  initiatives  and  responsiveness  to  help  others  perform  their  tasks  are  also 
evaluated. 

Decisions  and  actions  communicated  and  acknowledged.  Rate  the  extent  to  which  a 
course  of  action  is  announced  to  the  team  members  after  decision-making  input  is 
solicited  from  them.  Team  members  should  respond  verbally  or  with  appropriate 
adjustment  to  their  behavior  and  actions  to  clearly  show  that  they  understand  when  a 
decision  has  been  made  and  what  it  is.  Failure  to  announce  a  decision  may  confuse 
team  members  and  lead  to  uncoordinated  operation.  Note:  Due  to  time  constraints  in 
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certain  situations,  there  is  often  little  or  no  time  for  teams  to  make  inputs  to  a  decision. 
In  such  cases,  raters  should  focus  on  the  extent  to  which  decisions  are  announced  and 
acknowledged  verbally  or  through  coordinated,  pre-planned  action. 

Superior  Rating  (7) 

Team  members  communicate  with  each  other  and  their  patients  frequently.  Both 
senders  and  receivers  use  standard  terminology  and  unambiguous  language  for  all 
communications.  Senders  provide  clear,  concise  information.  Receivers  acknowledge 
nearly  all  messages  in  sufficient  detail  so  that  the  sender  can  verify  that  the  receiver 
understands  the  message.  Receivers  freely  ask  questions  to  clarify  their  understanding. 
Senders  pursue  feedback  when  no  response  is  forthcoming.  Senders  and  receivers 
integrate  check-backs  and  call-outs  into  clinical  situations.  Whenever  a  workload 
change  or  task  responsibility  transfer  (hand-off)  occurs,  the  change  is  communicated 
and  acknowledged  by  the  team.  In  the  case  of  communicating  with  patients,  team 
members  offer  frequent  updates  in  language  the  patient  can  understand.  Patients' 
questions  are  answered  as  completely  and  thoroughly  as  possible.  Discharge 
instructions  are  carefully  explained  and  supplemented  with  written  instructions. 

When  the  physician  is  formulating  a  diagnosis  or  a  situational  leader  is  making  a 
decision,  he  or  she  alerts  the  primary  care  team  and  seeks  suggestions  on  possible 
solutions  and  important  information  to  consider.  The  physician  or  situational  leader  is 
open  to  all  suggestions.  Team  members  respond  to  these  inquiries  with  sound, 
task-focused  discussions  and  clear  answers  provided  in  a  timely  manner.  Team 
members  raise  questions  on  diagnoses  and  treatment  plans  or  changes  to  plans  and 
actions.  Nearly  all  these  inquiries  surface  information  that  contributes  to  the  decision 
making  process.  The  physician  or  situational  leader  states  decisions  and  intended 
actions  and,  time  permitting,  explains  the  reasons  and  intent  Team  members 
acknowledge  the  decisions  with  a  clear  verbal  response  and  ask  questions  to  clarify  any 
confusion.  The  leader  answers  all  questions  in  a  positive,  straightforward  manner. 

Team  members  are  particularly  attentive  to  communicating  workload  changes  and 
information  essential  for  maintaining  situational  awareness.  All  team  members  seek 
assistance  and  information  from  others  in  performing  their  tasks.  Similarly,  all  team 
members  anticipate  the  task  needs  of  other  team  members  and  offer  relevant 
information  and  assistance.  Offers  of  assistance  cover  the  range  from  highly  skilled  to 
mundane  tasks.  Team  members  keep  one  another  informed  of  the  results  of  their 
activities  and  changing  task  responsibilities. 


Acceptable  Rating  (4) 
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Team  members  communicate  about  individual  patients  and  general  operational  issues 
as  required.  Standard  terminology  and  unambiguous  language  are  usually  used. 
Receivers  acknowledge  most  messages.  Receivers  generally  ask  questions  for 
clarification.  Senders  usually  pursue  feedback  when  no  response  is  forthcoming.  Team 
members  are  appraised  of  changes  to  significant  factors  and  task  responsibilities  during 
clinical  events.  Duties  are  specified  and  communicated  to  others.  Patients'  questions  are 
answered  in  understandable  language  and  updates  are  offered  intermittently. 

Discharge  instructions  are  brief  and  to  the  point  and  supplemented  with  written 
instructions. 

The  physician  reviews  available  patient  clinical  data  and  asks  team  members  for 
essential  information  to  arrive  at  a  diagnosis,  make  decisions,  and  formulate  a  plan  of 
action.  Time  permitting,  the  physician  offers  team  members  a  brief  explanation  his  or 
her  diagnosis  and  plans.  The  team  acknowledges  its  awareness  of  the  physician's 
decisions  and  directions.  Team  members  may  ask  questions  to  clarify  confusion.  The 
physician  or  situational  leader  answers  questions  clearly  and  quickly.  In  situations  of 
less  well-defined  diagnostic  or  problem  resolution  certainty,  the  physician  or 
situational  leader  alerts  the  team  to  the  need  for  more  extensive  information  exchange. 
Team  members  usually  respond  to  these  inquiries  with  brief  exchanges  that  may 
provide  previously  uncommunicated  information  or  insightful  suggestions.  The 
physician  or  situational  leader  listens  to  new  information  and  suggestions  without 
interruption  or  criticism  and  asks  for  clarification  as  necessary.  Team  members 
regularly  use  check-backs  and  call-outs. 

Team  members  tend  to  focus  their  information  gathering  and  exchange  on  patients 
within  their  individual  spheres  of  responsibility.  The  team  as  a  whole  maintains  a 
general  awareness  of  the  status  of  all  patients  being  cared  for  by  the  team.  The  team 
may  or  may  not  be  aware  of  the  general  status  of  operations  within  the  emergency 
department  Changes  to  situation  awareness  are  verbalized  across  the  team  as  the 
workload  changes.  Team  members  sometimes  seek  assistance  and  information  from 
others  in  performing  their  individual  tasks.  Similarly,  all  team  members  anticipate  the 
task  needs  of  others  and  offer  relevant  information  and  assistance  as  their  individual 
workload  permits.  Task  demands  dictate  the  level  of  information  exchange  and 
reciprocity  of  helping  behaviors.  That  is,  urgent  or  emergent  situations  reveal 
heightened  levels  of  verbal  exchange  and  greater  task  interaction  than  nonurgent 
situations. 

Veiy  Poor  Rating  (1) 

Team  members  may  fail  to  make  statements  regarding  critical  information. 
Non-standard  terminology  is  used  or  standard  terminology  is  used  inappropriately. 
Sender  messages  may  be  inappropriately  delayed  or  irregular  and  may  be  confusing. 
Receivers  usually  do  not  verbally  acknowledge  the  receipt  of  messages.  Receivers  do 
not  ask  questions.  Senders  do  not  pursue  feedback  when  no  response  is  forthcoming. 
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Check  backs  and  call-outs  are  seldom  or  never  used.  Changes  in  responsibilities  during 
an  event  are  often  not  communicated  and  may  result  in  confusion  over  who  has  a  task 
responsibility.  Information  may  be  incomplete  or  confusing.  Patients'  questions  are 
answered,  but  without  regard  to  the  clarity  or  understandability  of  the  answers. 
Generally,  updates  are  provided  to  patients  only  on  request;  they  are  not  offered 
voluntarily.  Discharge  instructions  are  cursory  and  written  instructions  may  not  be 
reviewed. 

The  physician  or  situational  leader  makes  decisions  without  seeking  inputs  from  other 
team  members  nor  alerting  the  team  that  a  decision  is  required  or  is  being  made. 
Decisions  and  intentions  of  the  physician  or  situational  leader  are  often  not  passed  on 
to  the  team.  [Note.  Decision  making  and  planning  are  done  by  one  individual  with 
little  or  no  discussion-an  observer  will  have  difficulty  noting  this  quality  for  "very 
poor"  teams  since  it  is  hard  to  detect  individual  decision  making.]  The  team  is  often  not 
aware  that  a  decision  has  been  made.  As  a  result  the  team  does  not  offer  suggestions 
and  inputs  to  support  decision  making  or  actions.  Team  members  almost  never  raise 
questions  about  plans,  actions,  or  changes  to  plans.  The  physician  or  situational  leader 
may  not  acknowledge  or  respond  to  questions,  or  may  abruptly  answer  them.  Two 
physicians  may  attempt  to  simultaneously  take  control  of  a  patient  when  control 
authority  has  not  been  negotiated. 

Team  members  infrequently  ask  for  team  assistance  with  tasks  even  when  they  are 
overloaded  to  the  point  of  nearly  failing  to  properly  execute  them.  The  team  generally 
does  not  offer  its  services  to  support  task  execution  by  other  team  members.  Team 
members  may  discourage  others  from  asking  questions  or  seeking  assistance  by  the 
tone  of  voice  they  use  or  by  failing  to  respond.  Team  members  may  take  uncoordinated 
actions  without  stating  intentions  or  expected  results. 
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Emergency  Medicine  Team  Dimension  4:  Execute  Plans  and  Manage 

Workload 


Overview 

This  rating  concerns  the  adequacy  of  carrying  out  plans  and  protocols  and  managing 
changing  levels  of  workload.  The  rating  begins  at  the  point  where  the  team  has  chosen 
to  carry  out  a  routine  treatment  plan  or  protocol,  or  has  completed  planning  activities 
required  for  an  unusual  situation  in  patient  care.  The  effectiveness  of  carrying  out  the 
course  of  action  for  an  individual  patient  and  executing  ongoing  plans  for  all  patients 
within  the  team's  area  of  responsibility  is  the  focus  of  this  assessment. 

One  dimension  of  the  rating  is  the  attentiveness  of  the  team  to  conducting  secondary 
triage  of  its  patients,  and  establishing  new  priorities  for  patient  care.  Reprioriatization 
is  necessary  as  the  conditions  of  patients  change  or  new  patients  arrive  who  require 
higher  levels  of  care.  This  periodic  reassessment  may  require  the  team  to  reenter  the 
decision  and  planning  cycles  for  one  or  more  patients. 

Evaluate  the  effectiveness  of  time  and  work  management  Rate  the  extent  to  which  the 
team  as  a  whole  avoids  being  distracted  from  essential  activities,  perceives  transitions 
in  workload  levels,  distributes  workload  among  team  members,  and  avoids  individual 
team  member  overload.  Evaluate  the  team's  performance  under  unusual  circumstances 
that  may  involve  high  levels  of  stress.  Assess  the  integration  of  technical  and 
managerial  strategies  for  contending  with  stressful  and  high  workload  situations.  The 
delegation  of  task  responsibilities  among  team  members  is  one  such  strategy  that 
should  be  considered  in  this  rating.  Another  strategy  is  the  prioritization  of  critical  and 
noncritical  duties. 

Rate  the  extent  to  which  the  team  uses  cross-monitoring  as  a  mechanism  to  avoid  errors 
and  reduce  risk.  Team  members  can  catch  each  other's  errors  or  risky  behavior.  Such 
redundancy  is  particularly  important  when  teams  are  fatigued,  stressed,  or  too  focused 
on  critical  tasks. 


Finally,  include  in  this  rating  an  evaluation  of  maintaining  situational  awareness.  Rate 
the  extent  to  which  team  members  keep  each  other  informed.  Information  reported 
includes  the  status  of  patients,  team  member  welfare,  and  significant  operational 
factors  within  the  emergency  department  such  as  equipment  status  and  institutional 
conditions. 


Superior  Rating  (7) 

The  team  remains  calm  and  imposes  the  maximum  amount  of  control  possible  over  the 
situation  given  the  available  time  and  internal  and  external  resources.  Each  team 
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member  immediately  takes  on  individual  workload  responsibilities  based  on  prior 
discussions  or  rehearsal.  Each  member  handles  his  or  her  own  responsibilities  and 
seeks  to  support  the  team  member  with  the  greatest  workload.  Team  members  are 
aware  of  workload  buildup  on  others  and  readjust  workload  by  assuming  emerging, 
unassigned  tasks.  Overloads  do  not  occur. 

Virtually  all  distractions  are  avoided.  Each  team  member  understands  precisely  what 
information  is  relevant  to  the  situation  and  what  information  is  simply  a  distraction.  If 
a  team  member  becomes  mildly  distracted,  other  team  members  remind  him  or  her  to 
focus  on  the  task.  Each  team  member  is  concerned  that  all  tasks  are  properly  executed 
and  checks  both  his  or  her  tasks  and  those  of  others.  When  mistakes  are  noted,  the  team 
member  making  the  error  is  quickly  informed  in  a  concise  manner  without  excessive 
formality.  The  person  in  error  accepts  this  feedback  as  a  normal  part  of  team 
operations.  All  duties  are  prioritized  and  noncritical  duties  are  delayed  until  low 
workload  periods  or  termination  of  the  event. 

The  team's  planning  horizon  is  proactive,  that  is,  always  "one  or  two  steps  ahead."  This 
is  evidenced  through  reprioritization  of  treatment  and  disposition  plans  for  some  or  all 

of  the  patients  within  the  team's  area  of  responsibility.  Replanning  may  occur  for  some 
patients. 

Team  members  maintain  situation  awareness  of  the  progress  of  their  patients  in  the 
emergency  department  process  (evaluation,  treatment,  and  disposition)  and  of  possible 
operational  impediments  to  that  progress.  All  changes  in  patient  conditions  and 
emergency  department  operational  status  are  verbalized  and  acknowledged.  Team 
members  also  maintain  situational  awareness  of  other  team  members'  well-being  (e.g., 
emotional  or  physical  conditions  that  may  affect  performance).  Team  members 
volunteer  information  on  themselves  so  that  others  may  be  supportive. 


Acceptable  Rating  (4) 


The  team  responds  to  problems  in  carrying  out  a  treatment  plan  or  protocol  without 
overreacting.  The  physician's  or  situational  leader' s  requests  for  information  are  met  by 
feedback  from  the  team.  The  team  generally  takes  actions  to  reduce  each  other's  task 
load  and  in  most  situations  provides  information  even  if  it  is  not  specifically  requested. 
The  leader  and  team  make  good  use  of  available  resources.  The  team  is  intense  but  only 
mildly  flustered  by  most  clinical  situations. 

Most  distractions  are  avoided.  The  team  performs  well  in  deciding  what  information 
and  activities  are  essential  to  the  clinical  event  Most  nonessential  information  is 
discarded  or  ignored.  Non-critical  duties  are  prioritized  and  delayed  until  low 
workload  periods  or  the  event  has  terminated.  Team  members  are  aware  of  individual 
team  member  workloads  during  each  phase  of  a  clinical  event  When  an  individual 
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team  member  appears  to  be  overloaded,  other  team  members  may  notice  and  take  on 
part  of  the  workload.  W orkload  sometimes  appears  poorly  distributed  and  may  require 
the  team  leader  delegate  responsibilities.  The  team's  replanning  and  reprioritization 
are  adequate  for  the  developing  situation. 

Team  members  often  check  each  other's  task  performance  for  errors.  An  individual 
making  an  error  is  informed  and  makes  the  needed  corrections.  Only  occasionally  are 
persons  in  error  annoyed  at  being  checked  and  corrected. 

Team  members  usually  provide  situation  updates  on  patient  and  emergency 
department  operational  status.  Obvious  changes  in  team  member  welfare  are  noted 
and  acknowledged  without  fear  of  sanctions. 

Very  Poor  Rating  (1) 


The  team  becomes  disorganized  and  flustered.  The  team's  requests  for  information 
elicit  inadequate  responses.  Team  members  may  focus  on  the  wrong  issues,  thus 
delaying  correct  response  to  the  problem.  The  team  focuses  on  only  one  solution  to  an 
event,  does  not  consider  other  plausible  alternatives,  or  chooses  an  inappropriate 
solution.  Lack  of  coordinated  actions  adds  to  the  confusion.  The  leader  and  team 
members  make  poor  use  of  available  resources  to  resolve  the  problem. 

The  team  is  easily  distracted.  The  team  is  unable  or  unwilling  to  decide  what  is 
important  and  relevant  to  the  immediate  situation.  There  is  little  prioritizing  of  duties 
or  actions.  Time  and  energy  may  be  wasted  on  low  priority  tasks.  Risks  to  patient 
safety  or  welfare  may  occur  as  the  team  focuses  on  minor  tasks  while  critical  tasks 
requiring  immediate  attention  go  unattended.  Neither  the  overloaded  party  nor  other 
team  members  take  voluntary  actions  to  eliminate  an  overload  condition.  The  team 
makes  little  or  no  effort  to  redistribute  task  responsibilities  as  situational  changes  occur 
and  new  tasks  arise.  Individual  team  members  experience  workload  overloads.  The 
team's  planning  horizon  is  "playing  catchup." 

Team  members  seldom,  if  ever,  check  each  other's  task  execution.  Team  members  are 
insulted  if  they  are  corrected  by  another  team  member. 

Team  members  do  not  routinely  provide  updates  on  patient  status  or  operational  status 
of  the  emergency  department.  Generally,  updates  are  provided  only  on  request;  they 
are  not  made  voluntarily.  Personal  welfare  problems  such  as  fatigue  or  lack  of 
attention  are  not  mentioned. 
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Emergency  Medical  Team  Dimension  5: 


Improve  Team  Skills 


Overview 

This  rating  concerns  the  ability  of  the  team  to  monitor  and  review  its  general 
performance,  evaluate  the  quality  of  its  work,  and  improve  its  teamwork  processes. 
Evaluate  the  team  on  spontaneous  and  planned  discussions  of  its  strengths  and 
weaknesses  with  respect  to  technical  skills  and  team  coordination.  Discussion  themes 
include  what  was  done  wrong,  what  might  be  done  better,  how  improvements  can  be 
made,  and  what  was  done  well. 

Rate  the  occurrence  of  situational  teaching  and  learning  that  occurs  as  events  unfold. 
One  example  is  a  physician  discussing  his  clinical  impressions  with  a  nurse  following 
his  or  her  examination  of  a  patient.  Another  is  an  EMT  explaining  to  a  physician  the 
proper  positioning  of  a  patient  for  intubation. 

Rate  the  frequency  and  adequacy  of  the  team's  process  reviews.  Process  reviews 
provide  a  mechanism  for  process  analysis  and  quality  improvement  and  focus  on 
methods  for  improving  team  effectiveness.  Rate  the  team's  use  of  the  five  team 
dimensions  to  systematically  examine  and  benchmark  its  performance.  This  requires 
recurrent  examination  of  both  technical  task  and  team  coordination  outcomes,  and 
answers  the  questions,  "What  is  the  team  doing?"  and  "How  well  is  the  team  doing  it?" 

Reviews  can  be  informal  or  formal.  Informal  reviews  tend  to  occur  close  to  the  time  of 
the  event  and  are  likely  to  be  limited  to  those  directly  involved  in  the  situation.  An 
example  is  two  team  members  discussing  how  to  improve  their  coordination  on  a 
procedure  they  have  just  completed.  Formal  reviews  can  occur  as  retrospective  audits 
or  case  conferences.  They  are  usually  scheduled  in  advance  and  provide  an  opportunity 
for  more  team  participation.  An  example  of  formal  reviews  are  educational  forums 
typically  taking  place  away  from  the  immediate  clinical  area.  They  are  usually 
scheduled  to  maximize  team  member  attendance 


Superior  Rating  (7) 


This  team  demonstrates  exceptional  attention  to  critiquing  and  improving  task  and 
process  skills.  Instruction  and  case  review  occur  whenever  opportunities  arise  and 
circumstances  allow.  Reviews  equally  address  task  issues  (i.e.,  quality  of  clinical 
interventions)  and  process  issues  (i.e.,  team  effectiveness).  The  team  dimensions  are 
integrated  into  all  critiques  of  team  performance. 
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Expert  coaching  and  teaching  are  provided  by  team  members.  Team  members 
demonstrate  an  interest  in  learning  and  teaching,  and  actively  seek  opportunities  for 
both. 

Formal  reviews  are  conducted  at  regularly  scheduled  intervals.  The  team  reviews 
clinical  outcomes  to  improve  the  quality  of  care  delivery.  Reviews  are  conducted  in  a 
professional  manner  with  an  emphasis  on  education  and  improvement  of  task 
performance.  Superior  teams  balance  clinical  case  review  with  team  performance 
review.  For  example,  a  clinical  case  conference  is  held  to  discuss  a  particularly  complex 
clinical  case.  In  addition  to  reviewing  the  clinical  course  of  events,  the  team  also 
considers  the  effectiveness  with  which  team  processes  such  as  decision  making, 
communication,  and  workload  distribution  were  managed  during  the  case. 

Members  are  comfortable  giving  and  receiving  critical  feedback;  they  freely  discuss 
how  the  team  is  functioning  and  make  recommendations  for  improvement.  All 
discussions  focus  on  behavior  and  information  and  avoid  accusatory  tones.  Innovation 
and  creative  problem  solving  are  encouraged.  Recommended  improvements  are 
implemented  and  outcomes  reevaluated  as  part  of  a  continuous  process. 


Acceptable  Rating  (4) 


Situational  teaching  and  learning  occurs  on  occasion,  usually  in  response  to  a  direct 
inquiry  from  another  team  member.  While  there  is  some  reluctance  to  reveal  one's 
limitations,  team  members  will  approach  certain  individuals  on  the  team  for  assistance 
with  their  learning  needs.  Formal  reviews  occur  on  an  intermittent  basis. 

When  members  engage  in  performance  review,  attention  is  paid  more  to  clinical  issues 
than  team  process  issues.  Case  reviews  focus  on  obvious  errors  and  identification  of 
methods  for  avoiding  these  errors  in  the  future.  If  one  member  of  the  team  is 
technically  weak,  others  can  compensate  and  take  advantage  of  the  situation  to  teach  a 
new  skill  or  improve  one  that  is  weak. 

Process  review  occurs  most  often  following  an  event  in  which  team  performance  was 
perceived  to  be  problematic  or  contributed  to  a  negative  patient  outcome.  The  team  is 
less  systematic  about  using  the  team  dimensions  as  a  reference  for  performance  review. 
Interactions  are  positive  and  remain  focused  on  behavior  and  decisions;  there  is  no 
"finger  pointing."  Reviews  are  aimed  at  correcting  the  immediate  problem  rather  than 
on  improving  general  team  performance. 


Very  Poor  Rating  (1) 
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Team  effectiveness  is  rarely  reviewed.  There  is  no  evidence  of  internalization  of  the 
team  dimensions  and  no  systematic  approach  to  performance  review.  There  is  little 
effort  to  learn  from  previous  actions.  Virtually  no  teaching  is  observed  even  when  clear 
opportunities  to  do  so  arise. 

Obvious  errors  are  corrected  with  little  or  no  concern  about  improving  future 
performance.  There  are  times  when  basic  procedures  are  ignored  or  violated. 
Opportunities  for  skill  development  are  overlooked.  Members  are  uncomfortable 
giving  or  receiving  feedback.  When  case  reviews  occur  (which  is  rare)  they  usually 
involve  finger-pointing  and  blaming. 
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